# HSE News Aggregator — Source Integration Research

Researched 2026-08-03 via live HTTP fetches (not guessed URL patterns). This is the source of truth behind `data/sources.json` — update both together if a source is re-verified.

| Source | RSS/API found (URL) | robots.txt notes | ToS/licensing notes | Recommendation |
|---|---|---|---|---|
| **EU-OSHA** | RSS confirmed: `rss-feeds/latest/news.xml`, `.../press_releases.xml`, `.../events.xml` (all HTTP 200, real items). | Wildcard disallow on `*/rss-feeds/latest/*`. | CC BY 4.0 — reuse/excerpt/republish with attribution + indication of changes. | **RSS**, but confirm intent with EU-OSHA before enabling at scale (robots.txt names the path even though the licence is open). |
| **EU Healthy Workplaces Campaign** | RSS confirmed: `.../rss-feeds/latest/news.xml` (HTTP 200, real content). | robots.txt **explicitly** disallows this exact path, plus a blanket `/*/feed` rule. | Same EU reuse framework, likely CC BY 4.0. | **Do not enable.** The explicit, specific disallow is a deliberate "don't automate this" signal that overrides the permissive licence. |
| **UK HSE** | Press/prosecutions RSS confirmed: `press.hse.gov.uk/feed/` (HTTP 200). Main site has no working RSS for safety alerts/publication updates — 404s on the obvious paths. | press.hse.gov.uk only blocks `/wp-admin/`; feed unblocked. | Crown copyright, reusable under **OGL v3.0** with required attribution; logo excluded. | **RSS** for press/prosecutions; **newsletter only** for safety alerts/publications. |
| **IOGP** | RSS confirmed: `iogp.org/feed/` (HTTP 200, real HSSE content). | Standard WordPress robots.txt, `/feed/` unblocked. | No explicit reuse licence found — standard copyright footer only. | **RSS**, but headline + short original summary + link-through only; never reproduce body text. |
| **IOSH Magazine** | None found. | Entire domain behind an active Cloudflare bot-challenge, including robots.txt itself. | Not reachable. | **Newsletter only** — bypassing the challenge would mean circumventing bot detection. |
| **UAE MOHRE** | None found for news. | — | — | **Unsupported.** |
| **Dubai Municipality** | `dm.gov.ae/feed/` returns HTTP 200 with an RSS content-type header, but the body was **empty on two independent fetches**. | Feed path unblocked. | Not evaluated. | **Do not integrate** until manually re-confirmed as returning real items. |
| **Saudi MHRSD** | RSS confirmed: `hrsd.gov.sa/rss.xml` (HTTP 200, real Arabic-language items). No English feed. | Unrestricted (`Allow: /`). | Government portal, no explicit reuse notice — treat as standard copyright. | **RSS**, Arabic-only; item links resolve to an internal backend hostname and need URL reconstruction from the slug. |
| **Abu Dhabi ADPHC (OSHAD-SF)** | WAF rejected the request outright. | Could not read. | Not evaluated. | **Unsupported / unverified.** Note: OSHAD is an Abu Dhabi (UAE) framework, not Bahraini. |
| **Bahrain Ministry of Labour** | Not tested in this pass. | — | — | **Needs dedicated follow-up** if Bahrain coverage is specifically wanted. |

## Honesty flags (not fully verified — do not treat as confirmed)
1. Dubai Municipality's `/feed/` — correct headers, empty body both times.
2. ADPHC/OSHAD (Abu Dhabi) — WAF-blocked, no live feed confirmed.
3. Bahrain's actual regulator — not identified or tested at all yet.
4. EU-OSHA's `publications.xml` and other secondary feeds on the [RSS hub page](https://osha.europa.eu/en/agency-rss-feeds) were not individually re-verified — only news/press/events were fetched directly.

## Runtime safety net
Beyond this table, `ingestion/fetch_feeds.py` re-checks `robots.txt` for every source at fetch time via `urllib.robotparser` and will skip + log any source whose feed path is disallowed — so a source becoming disallowed later (or a mistake in this table) doesn't silently bypass the policy.