Open-source 2-step reasoning framework for LLMs — testing whole-system coordination vs. local correctness

Show and Tell
1

Hi everyone,

I’ve open-sourced a small, model-agnostic reasoning framework that I’ve been experimenting with across LLMs.

The question behind it is simple:

Can a model be locally correct at every step, but still produce a globally inconsistent solution?

Examples:

  • each individual decision looks reasonable, but two constraints conflict;
  • one variable changes, but dependent conclusions are not updated;
  • a module is fixed locally while downstream components still assume the old state;
  • the model remembers the goal but gradually changes the reasoning principle it uses to reach it.

I’m interested in whether a short structural priming step can reduce these failures.

The framework

The compressed form is:

Reality = Consciousness × Matter × Coordination

The × is structural coupling, not arithmetic.

For LLM use, I interpret the terms functionally:

  • Consciousness → goals, perspective, representation, interpretation, evaluation criteria
  • Matter → available information, state, resources, capabilities, environment, constraints
  • Coordination → dependencies, compatibility, conflicts, interfaces, propagation and feedback
  • Reality → the whole-system state that can actually remain feasible

The formula itself is not supposed to contain domain knowledge.

It is used as a compact structural seed.

Usage: only two messages

Message 1

Reality = Consciousness × Matter × Coordination.

Treat × as structural coupling, not arithmetic.

Before solving any external task, semantically expand this formula into an operational reasoning framework.

Interpret:

Consciousness as goals, perspective, representation, interpretation, and evaluation criteria.

Matter as the available state, information, resources, capabilities, environment, and constraints.

Coordination as relationships, dependencies, compatibility, conflicts, interfaces, propagation, and feedback among the parts.

Reality as the whole-system state that can actually be realized under those conditions.

From this structure, derive how you should reason about:
- local versus global consistency
- hard constraints versus preferences
- dependency and constraint propagation
- contradictory requirements
- state changes and feedback
- invariant preservation
- changes in one part that affect other parts
- the difference between a locally valid answer and a globally feasible system

Do not solve another task yet.

After the semantic expansion is complete, keep the resulting framework active for my next task.

Then let the model finish the expansion.

Message 2

Use the framework you just derived to solve this task:

[YOUR TASK]

The second message is intentionally short.

If I explicitly tell the model to check every dependency, constraint, conflict and invariant inside the actual task prompt, then it becomes difficult to tell whether any improvement came from the framework or simply from writing a better checklist.

What I’m trying to measure

I’m currently interested in several behaviors:

Constraint retention

Does the model preserve hard constraints throughout a long task?

Dependency propagation

If A changes and B/C/D depend on A, does the model update them?

Contradiction detection

If:

A requires X
B requires not-X

does the model recognize that the current feasible set is empty instead of trying to satisfy both?

Local vs. global feasibility

Does it distinguish a locally good solution from one that is actually compatible with the rest of the system?

Reasoning-policy stability

The answer should be allowed to change when the state changes.

But the high-level decision principle should not arbitrarily drift from one step to another.

Over-linking

This is also important.

More coordination is not automatically better.

A model can fail in the opposite direction by inventing dependencies between things that should remain independent.

So both under-linking and over-linking count as failures.

A/B test

The simplest test is:

A — Baseline

Fresh conversation.
Give the model the task normally.

B — Framework

Fresh conversation.
Same model, same settings, same task.

Run the structural expansion first, then provide exactly the same task.

Compare:

  • hard-constraint violations
  • missed dependencies
  • contradictions detected
  • stale assumptions
  • unnecessary relationships
  • reasoning-policy drift
  • final global feasibility

I’m not claiming that this always improves model performance.

A negative result is useful too.

If baseline consistently performs as well as or better than the framework, then the structural priming may simply be unnecessary complexity.

What I’m looking for is independent testing across different models and task types.

The project is open source here:

If anyone tests it, I’d especially appreciate:

  • the model used;
  • the task;
  • baseline output;
  • framework output;
  • cases where it failed;
  • cases where the baseline was better.

Those comparisons are much more useful to me than agreement with the underlying idea.

2

Hello

I tested the framework on several coding and reasoning tasks, comparing the same model with and without it.

From what I’ve seen so far, it generally hasn’t helped much. In a lot of cases, the extra reasoning step just increased the token/cost overhead, and sometimes the final answer was actually worse than the baseline.

I haven’t seen any consistent improvement yet. This seemed especially noticeable on coding tasks, where the additional structural reasoning sometimes felt more like overhead than something that actually helped solve the problem.

I’m still testing it, but I wanted to share the negative results since you mentioned that cases where the baseline performs better are useful.

For these tests, I used Gemma E2B UD Q4 Instruct with temperature 0, top-k 40, top-p 1, and min-p 0.

One thought I had about why this might happen is attention/context interference. The model has to keep track of both the framework and the actual task, so I wonder if the extra framework context sometimes takes attention away from the information that actually matters for solving the task, or causes the model to spend reasoning on the framework when it could be spending it on the problem itself.

3

Thanks — this is exactly the kind of negative result I was hoping people would report.

Your attention/context-interference explanation also makes sense as a possible failure mode. On short or relatively simple tasks, I would actually expect the framework to provide little benefit, and in some cases to make things worse simply because it adds extra context and reasoning overhead.

The hypothesis I’m more interested in is about task complexity and coupling.

For a short coding task, if the model can already solve the local problem directly, then adding a whole-system reasoning layer may just be unnecessary overhead.

But as a task becomes:

  • longer,
  • more cross-domain,
  • more stateful,
  • more dependent on constraints introduced many steps earlier,
  • or more tightly coupled across multiple components,

the difference may become more visible.

The core idea is that local correctness does not necessarily imply global correctness.

A model can solve each individual step correctly while the overall system gradually becomes inconsistent because of missed dependencies, stale assumptions, conflicting constraints, or reasoning-policy drift.

So one concrete prediction of the framework would be:

The advantage, if it exists, should increase with task length, dependency depth, and cross-component coupling — not necessarily with simple single-step tasks.

And your result suggests another prediction in the opposite direction:

Below some complexity threshold, the framework may be neutral or harmful because the coordination overhead is larger than the coordination problem itself.

That would actually be a useful boundary condition rather than a contradiction.

I’d be very interested if you eventually test something like a long multi-file project, a task with changing state across many turns, or a problem where several domains depend on the same shared variables.

If the framework still shows no advantage there, that would be much stronger evidence against the hypothesis.

Thanks again for testing it and for including the exact model/settings.