Python graphlib 工作流实跑:校验循环依赖,追踪下游阻塞原因

Source

一份 AI 生成的测试计划写了三步:先读需求,再起草测试条目,最后评审。读需求可以先完成。但如果草稿要求“评审结束才能开始”,评审又要求“有草稿才能开始”,剩下两步会一直等。

这里用人工构造的 JSON 复现四种坏计划,脚本先校验和排序,再用几个本地函数模拟任务执行。把它放在执行前,可以直接提示哪条引用有问题、哪几个任务绕成了圈。

一、先统一“依赖谁”的写法

每条任务带一个 id 和一个 depends_on 数组。{"id":"draft","depends_on":["load"]} 表示 load 要在 draft 前完成。这个方向与 Python TopologicalSorter 的输入一致,字典的值是该任务的前置节点集合对应的可迭代对象。见 Python graphlib 文档。

输入情况 为什么要拦 提示示例
summary 引用不存在的 laod 执行器没有这条任务 summary references undefined task laod
两条都叫 load 用字典组图时会覆盖先前定义 duplicate task: load
draft 依赖自己 没有可以先完成的一端 draft -> draft
review 和 draft 互相依赖 完整计划无法排出顺序 review -> draft -> review

顺序也有讲究。先检查所有任务定义,再检查引用。这样允许 summary 写在 load 前面。检查引用很有必要,因为 TopologicalSorter 可以把仅作为前置项出现的节点加入图里。仅调用排序器时,拼错的 laod 可能会出现在排序结果中,直到交给业务执行器才出错。

二、先得到完整顺序,再决定是否运行

static_order() 返回可迭代结果。示例用 tuple(...) 把整个排序过程完成后才返回。外层拿到合法计划再启动运行,坏计划即使有一条独立任务能马上执行,也会整体拦住。

循环报错中包含一条找到的环。出现多个环时,修完提示里的那一个还要重新校验。两条都依赖 load 的任务之间可能有多种合法次序,因此回归断言检查“每条前置项排在它的任务前面”。如果团队要求可执行任务之间也有固定优先级,要单独定义排序规则。见 CycleError 与 static_order。

三、复制脚本验证

保存下面的代码为 demo.py,用 Python 3.9 及以上执行 python3 demo.py。Windows 可以尝试 py demo.py。实跑环境是 macOS ARM64、Python 3.13.3,全程使用标准库。传入任务 JSON 后,程序只调用自己登记好的本地处理函数。

import json
from graphlib import CycleError, TopologicalSorter


def read_plan(raw):
    tasks = json.loads(raw)
    if not isinstance(tasks, list) or not tasks:
        raise ValueError('expected a nonempty task list')
    graph = {
    
      }
    for task in tasks:
        if not isinstance(task, dict):
            raise ValueError('expected a task object')
        key, deps = task.get('id'), task.get('depends_on')
        if not isinstance(key, str) or not key.strip() or key != key.strip():
            raise ValueError('invalid task id')
        if key in graph:
            raise ValueError(f'duplicate task: {
      
        key}')
        if not isinstance(deps, list) or any(not isinstance(dep, str) for dep in deps):
            raise ValueError(f'invalid dependency list: {
      
        key}')
        graph[key] = tuple(deps)

    for key, deps in graph.items():
        for dep in deps:
            if dep not in graph:
                raise ValueError(f'{
      
        key} references undefined task {
      
        dep}')
    return graph


def plan_order(graph):
    try:
        order = tuple(TopologicalSorter(graph).static_order())
    except CycleError as exc:
        cycle = exc.args[1]
        raise ValueError('cycle: ' + ' -> '.join(cycle)) from exc
    return order


def run_plan(raw, handlers):
    graph = read_plan(raw)
    order = plan_order(graph)
    missing = sorted(key for key in order if not callable(handlers.get(key)))
    if missing:
        raise ValueError('no callable handler: ' + ', '.join(missing))
    status = {
    
      }
    for key in order:
        if any(status[dep] != 'ok' for dep in graph[key]):
            status[key] = 'blocked'
            continue
        try:
            handlers[key]()
        except ValueError:
            status[key] = 'failed'
        else:
            status[key] = 'ok'
    return status


def make_plan(*tasks):
    return json.dumps([{
    
      'id': key, 'depends_on': deps} for key, deps in tasks])


def expect_block(label, raw, reason):
    try:
        plan_order(read_plan(raw))
    except ValueError as exc:
        assert str(exc).startswith(reason), str(exc)
        print(f'{
      
        label}: BLOCK, {
      
        exc}')
    else:
        raise AssertionError(f'{
      
        label} was accepted')


if __name__ == '__main__':
    valid = make_plan(('summary', ['check', 'draft']), ('check', ['load']),
                      ('draft', ['load']), ('load', []))
    order = plan_order(read_plan(valid))
    positions = {
    
      key: i for i, key in enumerate(order)}
    graph = read_plan(valid)
    assert set(order) == set(graph)
    assert all(positions[dep] < positions[key] for key, deps in graph.items() for dep in deps)
    print('valid:', ' -> '.join(order))
    expect_block('missing', make_plan(('summary', ['laod'])), 'summary references undefined task')
    expect_block('duplicate', make_plan(('load', []), ('load', [])), 'duplicate task')
    expect_block('self', make_plan(('draft', ['draft'])), 'cycle:')
    cycle = make_plan(('load', []), ('review', ['draft']), ('draft', ['review']))
    expect_block('cycle with a ready task', cycle, 'cycle:')
    executed = []

    def record(key):
        return lambda: executed.append(key)

    handlers = {
    
      key: record(key) for key in graph}
    try:
        run_plan(cycle, handlers)
    except ValueError as exc:
        assert str(exc).startswith('cycle:') and not executed
        print('bad graph executed:', executed)
    else:
        raise AssertionError('cycle was accepted')

    def reject_draft():
        executed.append('draft')
        raise ValueError('draft data is incomplete')

    handlers['draft'] = reject_draft
    status = run_plan(valid, handlers)
    assert executed == list(order[:-1]) and status['summary'] == 'blocked'
    assert status['draft'] == 'failed' and status['check'] == 'ok'
    print('status:', status)
    print('executed:', executed)

四、实跑结果:坏计划零执行,失败不传到下游

valid: load -> check -> draft -> summary
missing: BLOCK, summary references undefined task laod
duplicate: BLOCK, duplicate task: load
self: BLOCK, cycle: draft -> draft
cycle with a ready task: BLOCK, cycle: review -> draft -> review
bad graph executed: []
status: {'load': 'ok', 'check': 'ok', 'draft': 'failed', 'summary': 'blocked'}
executed: ['load', 'check', 'draft']

这次把计划是否合法、任务是否成功拆成了两道检查。坏计划里有一条无依赖的 load,本来可以执行。程序先完整校验,bad graph executed: [] 证明一个处理函数都没被调用。这个断言能抓住“先调度所有可跑任务,再检查整张图”的行为差异。

合法计划中,load 后面有两个分支,check 正常结束,draft 报输入不完整。即使队列里下一项是 summary,它需要两项前置都成功,脚本直接标记为 blocked。实际执行列表中没有它。换成真实报告生成器时,可以把 failed 和 blocked 一起写进运行记录。前者需要查原始失败,后者通常应追到上游失败项。

还有一种部署差错是任务定义完整,但执行器没注册对应函数。代码在执行前检查所有处理函数,避免跑到一半才发现缺少实现。本例把 ValueError 作为已知的业务失败。实际接入时可定义更明确的业务异常,并记录异常详情。排序可以有多个结果,验收代码主要检查依赖关系和是否误执行,并保留当前运行的真实顺序。

五、顺着两层阻塞找到真正在失败的任务

链路是 draft → review → summary 时,汇总等的是 review,报“缺少验收条件”的是 draft。若日报只写“汇总没跑”,收到消息的人可能先去找评审同学,实际应该先补草稿输入。

把下面这段存成 trace_demo.py,与上面的 demo.py 放在一起,运行 python3 trace_demo.py。处理函数由执行侧绑定。示例把汇总结果写到临时目录,直接检查文件是否产生。

import json
from pathlib import Path
from tempfile import TemporaryDirectory

from demo import make_plan, plan_order, read_plan


class TaskFailure(Exception):
    pass


def execute(raw, handlers):
    graph = read_plan(raw)
    order = plan_order(graph)
    absent = [task for task in order if not callable(handlers.get(task))]
    if absent:
        raise ValueError("missing handler: " + ", ".join(absent))
    result = {
    
      }
    for task in order:
        waiting = [dep for dep in graph[task] if result[dep]["status"] != "ok"]
        if waiting:
            root_causes = {
    
      cause for dep in waiting for cause in result[dep]["root_causes"]}
            result[task] = {
    
      "status": "blocked", "waiting_for": waiting,
                            "root_causes": sorted(root_causes)}
            continue
        try:
            handlers[task]()
        except TaskFailure as exc:
            result[task] = {
    
      "status": "failed", "root_causes": [task], "error": str(exc)}
        else:
            result[task] = {
    
      "status": "ok"}
    return result


def main():
    raw = make_plan(("load", []), ("draft", ["load"]), ("audit", ["load"]),
                    ("review", ["draft"]), ("summary", ["review", "audit"]))
    with TemporaryDirectory() as directory:
        receipt = Path(directory) / "summary.txt"

        def draft():
            raise TaskFailure("missing acceptance criteria")

        handlers = {
    
      "load": lambda: None, "draft": draft, "audit": lambda: None,
                    "review": lambda: None, "summary": lambda: receipt.write_text("READY")}
        status = execute(raw, handlers)
        assert status["summary"]["waiting_for"] == ["review"]
        assert status["summary"]["root_causes"] == ["draft"]
        assert status["audit"]["status"] == "ok" and not receipt.exists()
        print(json.dumps(status, ensure_ascii=False))
        print("summary file after failure:", receipt.exists())
        handlers["draft"] = lambda: None
        fixed = execute(raw, handlers)
        assert all(info["status"] == "ok" for info in fixed.values())
        assert receipt.read_text() == "READY"
        print("summary file after fixed input:", receipt.read_text())


if __name__ == "__main__":
    main()
{"load": {"status": "ok"}, "draft": {"status": "failed", "root_causes": ["draft"], "error": "missing acceptance criteria"}, "audit": {"status": "ok"}, "review": {"status": "blocked", "waiting_for": ["draft"], "root_causes": ["draft"]}, "summary": {"status": "blocked", "waiting_for": ["review"], "root_causes": ["draft"]}}
summary file after failure: False
summary file after fixed input: READY

waiting_for 是当前任务的直接阻塞项,root_causes 给出原始失败节点。评审等 draft,汇总等 review,两条记录最后都指向 draft。独立的 audit 分支正常完成。补齐输入后重新执行,临时文件才写入 READY。

团队把这两类字段传给报告 Skill,就能同时写出“评审卡住”和“缺少验收条件”,减少人手工查链路的时间。这个处理只捕获明确约定的业务异常,意外程序异常继续向上抛出。示例第二次整图重放是教学验证,真实业务若涉及写外部系统,还需要另外约定重跑范围和幂等。