gazeCOM is a complete rewrite — in React and TypeScript — and a substantial expansion of gengaze (2025), the local, browser-based environment for saliency-guided image generation, introducing a new VLM mode, an extensive LLM system overhaul and weighted pools for workflows and prompts.
The original modular, self-contained tool for experimentation and hybrid control has grown from an interaction-focused paradigm into a fully-fledged “supergenerative” application, introducing a different approach to image processing. Alongside the classic interactive and algorithmic inputs — gaze, hand and cursor tracking, computer-vision prediction, and automated roamers — it now adds agentic, self-referential vision guidance.
The premise is unchanged: a stream of coordinates is gathered into a heatmap, reduced in turn to its center of mass, and fed into the generative workflow together with the image to be processed. The COM steers the region that gets cropped and, in composite mode, the direction in which the canvas drifts and expands (making gazeCOM, in effect, an automated outpainting machine).
With the introduction of the new VLM mode, at the close of a generation the returned frame can be sent back to a local vision model, whose reply — a single coordinate, drawn from an editable instruction — becomes the input point for the next cycle. The instruction is written in plain language: the most salient subject, an empty area, a weakly resolved region (and could itself be agent-driven). As the image changes, the next reading moves with it, so the system traces a self-rebalancing path through its own output rather than being influenced from the outside.
Where gengaze opened an exploration of neural space through gaze tracking in an enquiry about the limits of automation in the creative process, gazeCOM turns its focus onto machinic vision itself, becoming, in a sense, autopoietic.
With language and vision both delegated to Ollama, image generation to ComfyUI across modes, gazeCOM becomes, in effect, a substantially multi-agent-driven pipeline.
Now publicly available and open-source.
Find it here.
