From Sync to Async: Evolving the MCP Image Generation Service

System Architecture Reliability Engineering Model Context Protocol
Generated image div_001

Building a robust Model Context Protocol (MCP) service for image generation sounds simple on paper: receive a prompt, call an AI provider like DeepInfra or Together.ai, and return the generated image. However, the reality of long-running synchronous HTTP requests requires a much more sophisticated approach to both architecture and automated testing.

The Synchronous Trap

Generated image div_002

External image generation APIs are inherently synchronous and time-consuming. While a lightweight model might return an image in 3-5 seconds, high-quality models (like flux-dev) can take anywhere from 45 seconds to several minutes to process a single complex prompt.

In our initial implementation, the MCP tool generate_image was purely synchronous. It fired off a Python requests.post call and waited. This created a cascading failure of reliability issues:

  • Blocked Event Loops: The FastMCP/FastAPI backend was blocked, preventing health checks and concurrent tool discovery.
  • TCP & Gateway Timeouts: Proxies, load balancers, and reverse HTTP connections drop idle sockets well before a 5-minute generation task finishes, resulting in mysterious 504 Gateway Timeout errors.
  • Opaque User Experience: The client (and the AI Agent invoking the tool) had zero visibility into whether the image was actually generating or if the system had silently crashed.

Testing in the Dark: Diagnosing the Hang

Generated image div_003

The turning point occurred when our automated testing suites began to fail. Specifically, the test_generate_image_basic_call integration test was hanging indefinitely. Because the service was a black box, it was impossible to tell where the pipeline was stuck.

To illuminate the black box, we developed a highly granular, layered testing strategy:

  1. Connection Integrity: Added test_mcp_client_connection with strict 10-second timeouts to prove the MCP transport layer and tool discovery were functioning perfectly.
  2. Direct API Validation: Added test_deepinfra_api_direct to bypass the MCP layer entirely, proving that our API keys, payloads, and the external provider were healthy.
  3. Fail-Fast Tool Invocations: Wrapped our integration tests in asyncio.wait_for() with explicit timeouts (e.g., 60 seconds). A test that hangs forever is useless; a test that fails with TimeoutError provides actionable data.
"Testing shouldn't just verify that the happy path works; it must definitively prove exactly which layer failed when the unhappy path occurs."

With these diagnostic tests in place, the root cause became undeniable: the external API was taking longer than expected, and our synchronous blocking model was holding the entire system hostage without providing feedback.

The Async Job Lifecycle Solution

Generated image div_004

To fundamentally resolve the reliability issues, we architected an asynchronous job queue pattern directly within the MCP service. We decoupled the HTTP connection from the background image generation process.

1. The Job Manager

We introduced job_manager.py to act as an in-memory state machine. It assigns a UUID to every request and tracks its lifecycle through a strict JobStatus enumeration:

QUEUED -> STARTING -> GENERATING -> ENCODING -> COMPLETED (or FAILED)

Because external APIs don't stream progress, we implemented a time-based interpolation algorithm based on historical model averages. This allowed us to calculate an estimated completion percentage (e.g., smoothly progressing from 30% to 90% while the external API churned).

2. Non-Blocking MCP Tools

Instead of a single blocking endpoint, we exposed a suite of asynchronous tools to the MCP client:

  • generate_image_async: Accepts the prompt and returns a job_id in milliseconds.
  • get_image_generation_status: Allows the agent to poll the job by ID, returning the current state and progress percentage.
  • get_image_generation_result: Retrieves the base64 encoded image once the job enters the COMPLETED state.

3. Background Threading

To bridge the gap between the synchronous requests library and our asynchronous FastMCP server, we utilized asyncio.to_thread. This offloads the blocking network call to a separate OS thread, freeing the main event loop to handle status polling and health checks simultaneously.

We also implemented a progress_callback mechanism, allowing the background thread to safely cross the thread boundary via asyncio.run_coroutine_threadsafe and update the JobManager state in real-time.

Results and Reliability

Generated image div_005

The transformation was night and day. To prove the new architecture's resilience, we added test_concurrent_async_jobs. We rapidly bombarded the service with simultaneous image generation requests.

Under the old synchronous model, this would have locked the server completely. Under the new asynchronous model, the service instantly accepted all requests, queued them, and allowed the test suite to seamlessly poll their independent statuses as they generated in parallel background threads.

By treating image generation not as a function call but as a managed lifecycle, the mcp-image-gen service now provides a flawless, transparent, and highly reliable experience for any AI agent interacting with it.