The Collection Trigger Is Part of the Model

When we first trained compact context models, our sensor data came from moments when an application requested a decision from Context Decision. That dataset was useful for the question we had: could a neural model compress short windows of smartphone sensor measurements into a useful representation, run within mobile constraints and support different prediction heads?

Later, the question became broader. Could model outputs describe physical situations beyond those application-triggered moments? Could an activity prediction be interpreted on its own? Could the resulting intelligence improve an application's timing decisions in the real world?

At that point, the evidence contract changed.

The original dataset had not suddenly become bad. It remained useful for the experiment it was collected to support. But evidence that is adequate for one question is not automatically adequate for the next.

For mobile context models, the collection trigger is not merely a data-pipeline detail. It helps determine which situations enter the dataset, which situations remain underrepresented and which claims the resulting evidence can support. It belongs in the model specification.

A feasibility dataset is not a coverage dataset

Feasibility is not one question.

Architectural feasibility asks whether a model can meet defined computational and representational objectives. For on-device AI, this includes several practical questions:

• Can a model compress a short window of sensor measurements into a useful representation?
• Can it operate within mobile memory and inference constraints?
• Can multiple prediction heads consume the same representation?

These experiments help establish whether an architecture is technically viable. They do not, by themselves, establish that the model reliably covers every situation in which it may later be used.

For Context SDK's initial architecture experiments, we used decision-triggered sensor windows from the Context Decision pipeline. Broader collection was planned to support later questions about situational coverage. The existing collection path allowed us to investigate compact representations and runtime constraints without pretending that we had already validated the model across all intended sensing conditions.

The distinction matters because a single dataset may be asked to support several fundamentally different claims:

Success at one level does not establish success at the next. Each transition can require different data, different baselines and different validation methods.

The collection trigger defines the observed population

Context Decision collects sensor information around moments when an application requests a decision. That mechanism creates a particular sampling distribution.

The resulting observations are more likely to occur while the application is active. Other situations - such as a phone remaining in a bag during a journey - may be underrepresented.

Consider a phone on a train. Sensor observations collected when its owner opens an application may capture a different mix of movement, handling and phone placement than observations collected while the phone remains in a pocket. Both concern the same journey, but they do not necessarily describe the same sensing conditions.

A model evaluated only at app-opening moments has not thereby demonstrated that it can recognize the journey independently of those moments.

That does not make decision-triggered data inherently unsuitable. If a model will only be queried at the same kind of application decision point, those observations may be highly relevant to its intended operating conditions.

The problem changes when the intended use changes.

A reusable moment representation may need to describe physical situations independently of whether an application happened to request a decision. Activity recognition, phone placement and movement interpretation each introduce their own coverage and validation requirements.

In statistical terms, the distribution of observations conditioned on a decision trigger is not necessarily the distribution encountered across the model's broader intended uses. Application logic, operating-system execution opportunities and device-usage patterns all influence which physical situations enter the dataset.

The right question is therefore not whether a dataset is universally representative. It is:

Representative of which situations, under which collection conditions, for which intended use?

Why this matters for timing

Situational awareness is intended to help an application make better timing decisions: when to act, when to wait and when to get out of the way.

That makes the observation boundary especially important.

If a system observes the user's situation only when an application has already requested a decision, it learns from a selected set of moments. This may be appropriate for optimizing decisions made at that same trigger. It is not automatically sufficient for claims about the wider set of moments in which acting, waiting or remaining silent might matter.

The moments a system does not observe can be as important as the moments it does. A phone left untouched during a journey, for example, may represent a different operating condition from the moment the application is opened. Whether that difference matters depends on the application and the decision being supported—but the evidence must cover the conditions implied by the claim.

This is why situational awareness cannot be evaluated only as a property of a model. It must be evaluated as a relationship between the model, the collection mechanism, the intended operating population and the decision the application is trying to improve.

When an intermediate feature becomes a product output

The evidence contract can also change when the role of a model output changes.

A preliminary activity score may begin as an auxiliary input to a downstream predictor. The downstream model may learn to discount an uninformative score, although the score's contribution still needs to be evaluated.

But an auxiliary feature and a directly interpretable output are not the same thing.

Once a score is surfaced under a human-readable label - such as a particular transport mode - an engineer, product team or customer may reasonably interpret it on its own. That creates a different validation obligation.

An intermediate representation may be useful in an exploratory downstream experiment without each of its individual outputs being reliable enough to support independent conclusions. Moving from an auxiliary feature to a named activity prediction therefore requires renewed scrutiny of coverage, labels and evaluation - not simply another training run.

The relevant question becomes specific: what additional evidence does this new use require, and which conclusions remain outside the present evaluation scope?

Broader sampling is not neutral sampling

We subsequently expanded the experimental dataset by combining decision-triggered sensor windows with observations from a background-triggered collection path. This created more opportunities to examine situations that were relatively uncommon in the original data.

It also allowed us to inspect results separately across the two collection mechanisms instead of evaluating only one mixed population.

That separation matters. Background collection has its own operating-system constraints and selection effects. Combining two datasets does not produce an unbiased representation of human behavior. It produces a broader dataset assembled from two different observation mechanisms.

During the same research iteration, we also refined sensor transformations, reduced repetitive samples, strengthened evaluation checks and experimented with training regularization. Because several elements changed together, offline comparisons across the collection mechanisms cannot isolate the causal contribution of any single modification.

Nor does improved offline performance establish real-world reliability or incremental value for an application's decisions. Those remain separate questions requiring evidence appropriate to the claim.

Make the evidence contract explicit

The practical lesson is not to search for a dataset that is somehow representative of everything. It is to define the claim before judging whether the available data is adequate.

For a mobile context model, the evidence contract should make at least the following explicit:

• the model's intended use;
• the collection trigger or triggers;
• the sensing conditions and population those triggers are likely to cover;
• the claim the current evaluation supports;
• the conclusions that remain outside the evaluation scope;
• sample counts and key evaluation metrics separated by collection trigger.

Two operational habits follow directly from this work. First, store the collection trigger with every sensor window. Second, report relevant evaluation metrics by trigger, alongside the corresponding sample counts.

A result that holds for only one collection mechanism is still a result, provided the claim says so.

The collection trigger was not a neutral implementation detail. It shaped the observations available to the experiment and therefore the conclusions the experiment could support.

A dataset does not become inadequate merely because a model changes. It can become inadequate because the question we expect the model to answer has changed.