CrewAI Multi Tool Usage Single Trace Problem

[OnurYigitArpali]

I want to integrate langfuse to crewai. There is one crew having an agent that use multiple custom tools. After a single execution, i expect to create single trace by Langfuse. But it creates multiple trace entities for tools, generations, etc as seen in the images. I already tried the methods discussed in https://github.com/orgs/langfuse/discussions/6291. Also check the related documents, but i could not to create single trace.

from crewai.project import CrewBase, agent, crew, task
from langfuse import Langfuse
import openlit

from tools import (x,y)


@CrewBase
class AnalyzerCrew():

    agents_config = 'config/agents.yaml'
    tasks_config = 'config/tasks.yaml'

    @agent
    def planner(self) -> Agent:
        return Agent(
            config=self.agents_config['planner'],
            tools=[
                x,y
            ],
            verbose=True,
            allow_delegation=False,
            max_iter=2,
            max_rpm=15,
            max_execution_time=300
        )

    @task
    def research_planning(self) -> Task:
        return Task(
            config=self.tasks_config['research_planning'],
            agent=self.planner(),
            output_pydantic=ResearchPlan,
            output_file='research_plan.json'
        )

    @crew
    def crew(self) -> Crew:
        """Creates the CTI research planning crew - first stage of multi-agent workflow"""
        return Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential,
            verbose=True,
            planning=True,
            planning_llm="openai/gpt-4o-mini"
        )


_langfuse_client = Langfuse(
    public_key=config.public_key,
    secret_key=config.secret_key,
    host=config.host,
    debug=config.debug
)

openlit.init(
    tracer=client._otel_tracer,
    disable_batch=True
)


with _langfuse_client.start_as_current_span(
    name="Analysis Workflow"
) as span:
    # Initialize crew
    crew = AnalyzerCrew()
    print(f"✅ Crew initialized successfully")
    
    # Run analysis
    result = crew.crew().kickoff(
        inputs={'text': text, 'output_dir': output_dir}
    )

[KeepALifeUS]

The multiple traces issue is common with OpenTelemetry-style tracing. Here's how to consolidate:

Solution 1: Single Span Context

Ensure all operations share the same trace context:

from langfuse import Langfuse
from opentelemetry import trace

langfuse = Langfuse()

# Create a single parent trace
with langfuse.trace(name="Analysis Workflow") as trace:
    # Pass trace context to all operations
    result = crew.kickoff(
        inputs={"task": task},
        callbacks=[LangfuseCallback(trace_id=trace.trace_id)]
    )

Solution 2: Manual Span Hierarchy

with langfuse.trace(name="crew_execution") as parent:
    for agent in crew.agents:
        with langfuse.span(name=f"agent_{agent.name}", parent=parent):
            # Agent operations here
            pass

Solution 3: State-Based Tracing

Instead of relying on auto-instrumentation:

state = {
    "trace_id": str(uuid.uuid4()),
    "spans": [],
    "events": []
}

def log_event(name, data):
    state["events"].append({
        "trace_id": state["trace_id"],
        "name": name,
        "timestamp": time.time(),
        "data": data
    })

# After execution, send all events as single trace
langfuse.trace(
    trace_id=state["trace_id"],
    events=state["events"]
)

Root Cause

OpenLIT creates new traces for each tool call because it doesn't share context. Disable OpenLIT auto-tracing and manually instrument to maintain hierarchy.

[rifft-dev]

I think the key issue here is trace context propagation.

Your outer Langfuse span creates a parent span in your application code, but the CrewAI/tool/model calls will only attach to it if the same OpenTelemetry context stays active inside the CrewAI execution path.

When tools/generations show up as separate traces, it usually means one of these is happening:

1. OpenLIT is starting root spans for tool/model calls instead of child spans.
2. CrewAI execution is crossing an async/thread/task boundary and losing the current context.
3. The Langfuse/OpenTelemetry tracer provider is not the same one used by every instrumented library.
4. Custom tools are invoked outside the active parent span context.

The trace shape you probably want is:

Analysis Workflow
  crew.kickoff
    task: research_planning
      agent: planner
        tool: x
        tool: y
        generation: gpt-4o-mini
        
       
One debugging step I’d try: print the active OTel trace ID inside each custom tool. If that trace ID differs from the parent span’s trace ID, the context is being lost before or inside tool execution.

For custom tools, a practical workaround is to wrap each tool manually and force it to create spans under the current active context rather than relying entirely on auto-instrumentation.

I’m working on a multi-agent debugger called Rifft, and this parent/child trace shape is exactly the problem we’re trying to make easier to inspect. But even if you stay with Langfuse, I’d focus first on preserving one trace context from crew.kickoff() through agent/task/tool execution.