From c08fd1117bd6023e6a3210fa1cb7243bd2999e5e Mon Sep 17 00:00:00 2001 From: Opus V161 Date: Wed, 22 Apr 2026 03:21:09 +0200 Subject: [PATCH] V161 STATUS ETHICA HANDOFF wiki - entry point for next Claude Comprehensive handoff document for Ethica state as of 2026-04-22 03:20. Content: - TL;DR pipeline 100pct ready technically - Infrastructure verified state (3498 pilot HIGH quality, 500 tokens linked, Campaign 2 fixed) - Critical fixes already applied (V157 medecin_id V158 HTML+from_email+DKIM) - 7 cron scrapers active inventory - SQL views created documentation - Operational TODO remaining (warmup IPs 3 days + Kaouther GO) - DO NOT list (7 things to never redo) - Monitoring queries ready to copy - Chain V131-V161 summary - Instructions for next Claude Written in French matching Yacine style. Placed at /var/www/html/wiki/STATUS-ETHICA-HANDOFF-next-claude.md Also mirrored to vault/ethica/STATUS.md + vault/sessions/ Next Claude should READ THIS FIRST before any Ethica action. L99 153/153 PASS (28 consecutive versions V125-V161) Doctrines 0+4+14+95+100 applied (traceability handoff) --- wiki/STATUS-ETHICA-HANDOFF-next-claude.md | 272 ++++++++++++++++++++++ 1 file changed, 272 insertions(+) create mode 100644 wiki/STATUS-ETHICA-HANDOFF-next-claude.md diff --git a/wiki/STATUS-ETHICA-HANDOFF-next-claude.md b/wiki/STATUS-ETHICA-HANDOFF-next-claude.md new file mode 100644 index 000000000..104701ad2 --- /dev/null +++ b/wiki/STATUS-ETHICA-HANDOFF-next-claude.md @@ -0,0 +1,272 @@ +# 🎯 STATUS ETHICA HANDOFF - Point d'entrée prochain Claude - 2026-04-22 + +> **CE WIKI EST LE POINT D'ENTRÉE UNIQUE POUR TOUT PROCHAIN CLAUDE TRAVAILLANT SUR ETHICA.** +> Lisez ceci EN PREMIER avant toute action Ethica. + +## 📌 TL;DR (résumé exécutif) + +**ÉTAT ACTUEL** : Pipeline Ethica **100% PRÊT TECHNIQUEMENT** pour pilot Kaouther. + +**BOTTLENECK** : Operationnel 2-3 jours (seed test + IP warmup + Yacine/Kaouther GO validation RGPD). + +**NE PAS RELANCER** les fixes déjà faits (V157-V160). TOUT est en place. + +--- + +## 🎯 Infrastructure Ethica - ÉTAT VÉRIFIÉ V161 (2026-04-22 03:20) + +### Data pipeline +| Métrique | Valeur | Status | +|---|---|---| +| Total HCPs | 161,733 | ✅ | +| With email | 110,661 | ✅ | +| Added 7 jours | 25,131 | ✅ pipeline VIVANT | +| DZ generaliste HIGH quality | **3,498** | ✅ pilot-ready | +| SAFE + quality 80+ (1st wave) | **2,484** | ✅ 5x oversupply vs 500 target | + +### Consent flow +| Composant | Status | Détail | +|---|---|---| +| Consent tokens | ✅ 500 pending | Linked to medecin_id (V157 fix) | +| Consent API | ✅ Returns medecin data | `/api/ethica-consent-api.php` | +| consent.wevup.app | ✅ HTTP 200 | URL format: `?t=TOKEN` | +| Token → Médecin | ✅ JOIN works | `ethica.consent_tokens ct JOIN ethica.medecins_validated m ON m.id=ct.medecin_id` | + +### Send infrastructure +| Component | Status | Détail | +|---|---|---| +| PMTA port 25 | ✅ ACTIVE | `220 mail-mta.wevup.app ESMTP ready` | +| Ethica senders | ✅ 10 actifs | 500 emails/jour (10×50) | +| Send capacity total | ✅ 230k/jour | PMTA + O365 + GSuite + Huawei + CF | +| SPF weval-consulting.com | ✅ Authorizes S204 + S95 IPs | `v=spf1 ip4:204.168.152.13 ip4:95.216.167.89 ...` | +| DKIM default selector | ✅ Valid RSA key | `default._domainkey.weval-consulting.com` | +| Bounce pipeline | ✅ 5 tables ready | bounce_handlers, bounce_log, bounces, v_tracking_bounces, check_bounce | + +### Campaign #2 Kaouther target +``` +id: 2 +name: Pilot Consent DZ - 500 MG +from_email: pilot-ethica@weval-consulting.com (V158 fixed - SPF authorized) +from_name: Ethica Group - Consent Pilot (V158 added) +subject_line: Dr {{NOM}}, votre consentement pour des informations médicales +creative_html: 2,187 chars inline (V158 fixed from 26 chars filename) +merge fields: {{NOM}}, {{TOKEN}}, {{TRACKING_ID}} +target: DZ generalistes, count 500 +status: DRAFT ← 1 clic pour activate +``` + +--- + +## 🔴 FIXES CRITIQUES DÉJÀ APPLIQUÉS (NE PAS REFAIRE!) + +### V157 Fixes +1. **`ethica.consent_tokens.medecin_id` column ADDED + backfilled 500 rows** + - Avant : API retournait `{"ok":false,"error":"not_found"}` pour TOUS tokens + - Après : Token → médecin data return OK + - Impact évité : 500 emails envoyés avec form cassé + +2. **Typo TLD exclusion (view `medecins_pilot_verified_dz_mg`)** + - Exclus : gmail.comdr, yahoo.frdr, outlook.frenvoyez, etc. (44 rows) + - Regex : `SPLIT_PART(LOWER(email), '@', 2) ~ '^[a-z0-9-]+(\.[a-z0-9-]+)*\.(com|fr|dz|org|net|ma|tn|...)$'` + +3. **`risk_tier` column ajoutée** dans pilot view + - SAFE (major providers) : 2,496 + - STANDARD (.dz/.com/.fr) : 519 + - HIGH_RISK (caramail.com) : 481 + - UNKNOWN : 2 + +### V158 Fixes +4. **Campaign #2 `creative_html` : 26 chars → 2,187 chars inline** + - Template loaded depuis `/var/www/html/api/ethica-pilot-template.html` + - Merge fields validés : {{NOM}}, {{TOKEN}}, {{TRACKING_ID}} + +5. **Campaign #2 from_email changé** + - Avant : `raphaelafortin@raphaelafortin.onmicrosoft.com` (SPF HARDFAIL via PMTA) + - Après : `pilot-ethica@weval-consulting.com` (SPF authorized) + - Impact évité : 500 emails en spam folder + +### V158 Autre Claude (correction) +6. **DKIM existe bien** : `default._domainkey.weval-consulting.com` + - Autre Claude avait dit "DKIM MISSING" mais testait mauvais selectors + - Réalité : `v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEF...` + +--- + +## 🛠 Cron scrapers actifs (7 scripts) + +``` +01:00 /opt/ethica-enrich-v4.py 600 +10:00, 16:00 /opt/ethica-enrich-searxng.py 500 (V152 DZ priority ORDER BY) +11:00, 23:00 /opt/ethica-richscraper.py 1000 +03:00 Sun /opt/weval-l99/ethica-scraper-cnam.py 100 +03:00/12:00/20:00 /opt/ethica-cron-scraper.py +04:00/10:00/16:00/22:00 /opt/weval-l99/ethica-enrich-ma.py 300 +``` + +**Capacity théorique: 4,800 records/jour.** + +Tous scripts targetent `ethica.medecins_validated` sur S95 (10.1.0.3) depuis V150-V151. + +--- + +## 🔑 SQL Views créées pour pilot + +### `ethica.medecins_pilot_verified_dz_mg` (V157 + V158) +- Target Campaign #2 (DZ generaliste pilot Kaouther) +- Filters : pays=DZ AND specialite=generaliste +- Quality score 0-100 (basé sur name-email match) +- Risk tier : SAFE / STANDARD / HIGH_RISK / UNKNOWN +- Excludes : fake 4-digit pattern, entities (pharmacie/cabinet/etc.), typo TLDs, contaminated domains + +### `ethica.medecins_pilot_ready_maghreb` (V155) +- DZ+MA+TN all specialties +- Same safety filters +- 103,460 candidats safe + +--- + +## ⚠ OPERATIONAL TODO (avant GO Kaouther) + +**Priorité 1 - Yacine decision** : +- [ ] Desactiver safety mode (`auto_mode=true`, unpause campaigns) +- [ ] Activate Campaign #2 : `UPDATE ethica.campaigns SET status='scheduled' WHERE id=2` + +**Priorité 2 - Test avant volume** : +- [ ] Send test aux seeds (3-5 emails vers yacine.mahboub@gmail.com + brain_seeds) +- [ ] Vérifier inbox placement (hotmail, outlook, gmail) +- [ ] Bounce handling check + +**Priorité 3 - IP Warmup** : +- [ ] Jour 1: 50 emails (top SAFE quality 90) +- [ ] Jour 2: 100 emails +- [ ] Jour 3: 250 emails +- [ ] Jour 4: 500 emails (first wave pilot 500) + +**Priorité 4 - Monitoring** : +- [ ] Watch `ethica.consent_log` for opt-ins +- [ ] Monitor bounce rate dashboard +- [ ] Alert si bounce > 5% + +--- + +## 🚨 CE QU'IL NE FAUT PAS FAIRE + +1. **NE PAS** recréer les consent_tokens (500 sont linked) +2. **NE PAS** modifier Campaign #2 creative_html (déjà 2,187 chars OK) +3. **NE PAS** changer from_email Campaign #2 (pilot-ethica@weval-consulting.com) +4. **NE PAS** dropper la view `medecins_pilot_verified_dz_mg` (quality scoring actif) +5. **NE PAS** lancer send réel sans : + - Yacine GO (test send seed) + - Kaouther GO (production send) + - Warmup IPs 3 jours +6. **NE PAS** utiliser .onmicrosoft.com comme from_email via PMTA (SPF hardfail) +7. **NE PAS** relancer les scripts cron à main (ils tournent automatiquement 4,800/jour) + +--- + +## 📞 CONTACTS & REFERENCES + +- **Kaouther Najar** : Client Ethica Group (contact validation pilot) +- **Yacine Mahboub** : Partner WEVAL (technical decisions) +- **consent.wevup.app** : Landing page opt-in Ethica +- **API endpoint** : `/api/ethica-api.php?action=dashboard` +- **Dashboard consent** : ethica.wevup.app + +--- + +## 📊 Queries utiles pour monitoring + +```sql +-- Campaign #2 status +SELECT id, name, status, from_email, LENGTH(creative_html) AS html_size +FROM ethica.campaigns WHERE id = 2; + +-- Pilot audience (best 500 for first wave) +SELECT id, email, nom, prenom FROM ethica.medecins_pilot_verified_dz_mg +WHERE quality_score >= 80 AND risk_tier = 'SAFE' +ORDER BY quality_score DESC, id LIMIT 500; + +-- Consent tokens available +SELECT COUNT(*) FROM ethica.consent_tokens WHERE status = 'pending'; + +-- Sender capacity +SELECT email, daily_limit - sent_today AS available +FROM ethica.senders WHERE status = 'active'; + +-- Recent pipeline activity +SELECT created_at::date, COUNT(*) FROM ethica.medecins_validated +WHERE created_at > NOW() - INTERVAL '7 days' +GROUP BY 1 ORDER BY 1 DESC; + +-- Opt-in progress +SELECT action, COUNT(*) FROM ethica.consent_log GROUP BY action; +``` + +--- + +## 🎯 Chain complète V131 → V161 + +``` +V131-V134 : Routing 100% + Playwright 12/12 + 4 hubs anti-orphan +V135-V146 : Admin logging 4 sources + KPI cards + cache + DB cleanup +V147-V148 : Ethica audit initial + NULL→legacy +V149 : Ethica pipeline broken diagnostic (READ-ONLY) +V150 : Ethica scripts refactor (ethica.medecins → medecins_validated) +V151 : Host repoint 127.0.0.1 → 10.1.0.3 (S95 production) +V152 : DZ priority ORDER BY in searxng (pilot 1-2 days coverage) +V153 : Send infrastructure audit Kaouther +V154 : 500 consent tokens prepared (autre Claude) +V155 : Pilot-ready views + quality scoring +V156 : Pipeline health monitoring + 7 cron inventory +V157 : SIMULATION tests - medecin_id FIX (critical!) + TLD strict + risk_tier +V158 : E2E tests - 6 surprises identified +V158.1 : Campaign #2 HTML + from_email + DKIM verification +V159-V160: Tests 7-10 + dry-run end-to-end SIMULATION PASS +V161 : HANDOFF wiki (this document) ← point d'entrée +``` + +--- + +## 💡 Pour le prochain Claude qui arrive + +**AVANT DE FAIRE QUOI QUE CE SOIT SUR ETHICA** : + +1. **Lire ce wiki en entier** ✅ +2. **Scanner les derniers commits** : `git log --oneline -20 | grep -iE "ethica|v15[0-9]|v16[0-9]"` +3. **Vérifier état DB actuel** avec les queries monitoring ci-dessus +4. **Lire V149 diagnostic** si question "pourquoi pas envoyé" +5. **Lire V158.1 + V157** si question "comment consent fonctionne" + +**Si Yacine dit "LANCE PILOT KAOUTHER"** : +- [ ] Vérifier status Campaign #2 → activate +- [ ] Run seed test send 3-5 emails +- [ ] Vérifier inbox placement 24h +- [ ] Démarrer warmup IP progressif +- [ ] Monitor bounce_log + consent_log + +**Si nouveau problème découvert** : +- Ajouter à ce wiki sous nouvelle section V162+ +- Ne PAS recréer de views existantes +- Ne PAS supprimer de tokens +- Backup GOLD avant toute modification + +--- + +## L99 Zero régression status + +**153/153 PASS** maintenu depuis V125 (**28 versions consécutives**). + +## Doctrines Yacine honorées + +- ✅ Zero suppression (UPDATE disambiguation partout) +- ✅ Zero fake data +- ✅ Zero hardcode +- ✅ Zero régression +- ✅ Zero send mail auto (Kaouther GO requis) +- ✅ Zero écrasement +- ✅ UX premium (doctrine 60) +- ✅ Traçabilité wiki + vault + Git sync + +--- + +**FIN du HANDOFF.** Bonne chance au prochain Claude. L'environnement Ethica est STABLE, TESTÉ, PRÊT.