← Engineering blog

Stop reinstalling the tools before every task

A reporting agent starts a fresh machine, installs jq to inspect JSON, then finally gets to the data. The next job repeats the installation. In a short sandbox session, that’s a lousy place to spend time: the inputs change, but the tools don’t.

Mainbrella’s custom images move that setup into a build, so later machines can start with the tools installed. Each machine still gets its own filesystem changes. The image can prepare tomorrow’s reporting job; today’s report needs somewhere else to live.

Check whether the tools are already there

Start with authenticated GET /containers. Its imageCatalog lists the runtime images available in that deployment. If a returned node or python image supplies what the job needs, use its ID as catalogId when creating the container. Maintaining a custom build for tools already in the catalog is extra work for no benefit.

If something is missing, read GET /images for buildsEnabled, limits, usage, and your owned images. Custom builds depend on deployment configuration. A working catalog doesn’t establish that the build service is enabled; check before submitting a recipe.

Put the repeatable setup in the recipe

For a job that repeatedly needs jq, the Dockerfile can be this small:

FROM mainbrella:base
RUN apt-get update \
    && apt-get install -y --no-install-recommends jq \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /workspace

This adds a tool to our base environment. Keeping apt-get update and installation in the same step follows Docker’s guidance for avoiding stale package indexes. It’s an illustrative recipe, not a recorded hosted build. It also leaves the package version unpinned, so rebuilding later can select a newer jq.

Launching a ready image reuses the built files. To use a changed Dockerfile, submit another build and select its new image for future machines.

Keep the recipe in source control. Put shared tools and stable dependencies there, then supply each job’s input when it runs. Baking customer data or an account credential into an image would give every machine launched from it the same bytes.

The build contract accepts a multipart name, text dockerfile, and optional .tar.gz context. The first instruction must be FROM mainbrella:base, after optional comments, with one build stage. Dockerfiles are limited to 16 KiB and compressed contexts to 512 KiB. Retain /bin/sh and GNU coreutils for command and file operations, plus bash and tmux for terminal compatibility.

Accepted, built, and ready are different moments

POST /images returns HTTP 202 with an image record. Save its id and inspect GET /images/{id}. The successful path is queued → building → publishing → ready; failures end in failed, with output at GET /images/{id}/logs.

After Docker finishes, the publisher uploads the image, records its content digest, and deploys the runtime’s image map before reporting ready. A successful Docker build alone hasn’t made the image available to launch.

We also separate the jobs: the Dockerfile’s commands execute in a disposable build job, while registry and API callback credentials go to the publication job. That job loads and publishes the artifact without executing it. A customer recipe shouldn’t need our publishing credentials to install a package.

The workflow gives docker build 300 seconds. Queueing, compatibility checks, and publication add time; five minutes isn’t a request-to-ready promise. Current account defaults are ten builds per UTC month and three saved images. Use the API’s returned limits.

Only after ready, create a container with {"imageId":"<the saved id>"}. Omit catalogId; the two selectors are mutually exclusive. The image changes installed software, while the selected size and lifecycle still govern resources and deadlines.

The next machine gets the tools, not the report

Check jq --version on the new machine, then upload /workspace/input.json containing [12,17,13]. This tiny reporting command has an answer you can check without a model:

jq -r '["total"], [add] | @csv' \
    /workspace/input.json > /workspace/report.csv

Download the file through the file API; it should contain a total header and the value 42. The agent setup workflow exercises binary transfer too. Check that loop before asking an agent to process valuable input.

Starting another machine from the same image gives it jq, but no copy of that CSV. Writes in a running container don’t update its source image. The two launch paths below start with the same tools; only the first job produces and exports a report.

One prepared image containing jq launches two separate machines. Job A writes report.csv and exports a copy outside the machine before stop. Job B starts with jq but no report.csv. Neither job writes back to the image.
What you keepWhat it carries forward
Custom imageTools and files included at build time.
Saved workspaceAn explicitly captured filesystem, restored into a fresh machine. No running processes or RAM.
Exported reportThe task’s output, stored outside the machine.

Saved workspaces have their own quotas and retention. Deleting an image leaves already-running containers in place; active builds cannot be deleted. Neither operation exports the report.

For the real reporting job, export the CSV and check its rows before stopping that exact container generation. Individual HTTP file transfers are limited to 1 MiB, so larger reports need a planned export route. Keep the Dockerfile with the reporting code and the completed report with the data it explains.