Pathway

A multi-agent system that turns life-story signals into a causal career roadmap.

---

Overview

Pathway ingests a person's career signals ( CV, scraped articles, podcast & video transcripts ) and produces a causal directed acyclic graph (DAG) of milestones that explains how they reached each goal. A coordinated set of specialized agents then critiques, repairs, and enriches the graph through interactive interview, counterfactual reasoning, and confounder analysis.

The system is designed for three audiences:

Audience	What Pathway delivers
Mentees & job-seekers	A personalised, causally coherent roadmap they can follow.
Mentors & career coaches	A structured artefact to discuss trade-offs and missing steps.
Researchers	A reproducible multi-agent testbed for causal reasoning on biographies.

---

System architecture

flowchart LR
    subgraph S[Sources]
        A1[CV PDF]
        A2[Web articles & blogs]
        A3[YouTube videos / podcasts]
        A4[Free-form text]
    end

    subgraph I[Ingestion]
        B1[Scrape and save]
        B2[Transcribe audio]
        B3[Normalise paragraphs]
        B4[Resume vision parser]
    end

    subgraph G[Roadmap generation]
        C1[Career summary builder]
        C2[Goal and milestone extractor]
        C3[Target format normaliser]
    end

    subgraph K[Causality layer]
        D1[Causal DAG extractor]
        D2[Counterfactual agent]
        D3[CVA confounder agent]
        D4[Gap filler agent]
        D5[Structure verifier]
    end

    subgraph U[Interactive layer]
        E1[Streamlit roadmap explorer]
        E2[Flask interview server]
        E3[Vote and refinement loop]
    end

    subgraph E[Evaluation]
        F1[User simulator]
        F2[LLM judge]
    end

    A1 --> B4
    A2 --> B1 --> B3
    A3 --> B2 --> B3
    A4 --> B3
    B3 --> C1
    B4 --> C1
    C1 --> C2 --> C3 --> D1
    D1 --> D2
    D1 --> D3
    D1 --> D4
    D1 --> D5
    D2 & D3 & D4 & D5 --> E1
    D1 --> E2 --> E3 --> D1
    E1 --> F1 --> F2

Loading

A deeper view of the agent topology lives in docs/architecture.md.

---

Repository layout

Pathway/
├── pathway/                      Main Python package
│   ├── agents/                   Specialised agents (orchestrator, safety, gap filler, ...)
│   ├── causality/                DAG extractor, causal engine, pipeline
│   ├── ingestion/                Scrapers, transcriber, CV-to-summary, normaliser
│   ├── ui/                       Streamlit roadmap explorer
│   ├── interview_server/         Flask-based interactive interviewer
│   └── evaluation/               LLM judge and user-simulator
├── scripts/                      Thin command-line entrypoints
├── data/                         Input persons, generated outputs, backups
├── docs/                         Architecture, pipeline, agents, interview, evaluation
└── tests/                        Test harness (to grow)

A detailed file-by-file reference is in docs/agents.md.

---

Quickstart

1. Install

git clone https://github.com/<your-org>/Pathway.git
cd Pathway
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .

2. Configure API keys

Copy the example environment file and fill in the keys you have access to:

cp .env.example .env
$EDITOR .env

Variable	Used by
OPENAI_API_KEY	CV parser, milestone generator, interview agents
GROQ_API_KEY (a.k.a. groq_api_key)	Causal DAG extractor, safety moderator, summariser
OPENROUTER_API_KEY	User simulator
GEMINI_API_KEY	LLM judge (optional)

3. Add a person

Put a profile name into data/persons/names/ (one text file per person, with one URL per line of public material to ingest) and, if available, the resume PDF into data/persons/linkedin/<Name>.pdf.

4. Build a roadmap end-to-end

python -m scripts.build_roadmap

This runs: scrape → transcribe → normalise → career summary → milestone extraction → causal DAG construction. Outputs land in data/outputs/ with timestamped backups in data/backups/.

5. Explore the roadmap

streamlit run pathway/ui/app.py

The Streamlit explorer lets you inspect the DAG, vote on milestones, request gap-filling suggestions, and run counterfactual or confounder analyses.

6. Run an interactive interview (optional)

python -m scripts.run_interview_server
# then open http://127.0.0.1:8000/<interview_id>/<session_id>

7. Evaluate

python -m scripts.run_evaluation

---

Documentation

Document	Topic
docs/architecture.md	Component topology, message flow, agent orchestration
docs/pipeline.md	Ingestion → causal DAG, with stage-by-stage diagrams
docs/agents.md	Reference card for every specialised agent
docs/interview.md	Flask interview server, prompt design, voting protocol
docs/evaluation.md	Judge metrics, simulator personas, reproducibility

---

Concepts

A goal is a discrete career objective (e.g. Research Scientist at QCRI). A milestone is a prerequisite step (degree, project, publication, network event). Edges encode typed causal links:

Type	Meaning
direct_cause	A directly causes / enables B
indirect_cause	A influences B through intermediates
prerequisite	A must complete before B can start
enables	A makes B possible but is not required
supports	A helps B; B can still happen without A
mutual_reinforcement	A and B strengthen each other
inhibitory	A reduces the likelihood of B
conditional	A causes B only under certain conditions
temporal	A must precede B in time

Causal-DAG operations are implemented over networkx.DiGraph in pathway/causality/.

---

Contributing

Contributions are welcome. Please read CONTRIBUTING.md and open an issue describing the change before submitting a PR.

---

License

Released under the MIT License.