ContextSDK logo in purple
Products
ContextDecisionContextPush
Solutions
Solutions
Teams
DevelopersHeads of ProductMarketers
Industries
GamingEntertainmentHealthSocial MediaDating
Use cases
Dynamic Product ExperiencePush Notification Open Rate
Teams
DevelopersHeads of ProductMarketers
Industries
GamingEntertainmentHealthSocial MediaDating
Use Cases
Dynamic Product ExperiencePush Notification Open Rate
Resources
Resources
Case StudyNewsletterBlogDemo AppDocs
Company
ContactCareersPressPrivacy & Security
LoginContact us

Unifying Data Models Across a Heterogeneous Stack

March 7, 2024

In the realm of software development, ensuring that different components of a product interact seamlessly with a consistent data model is crucial. For ContextSDK, this is especially important since data is generated and processed by applications written in multiple programming languages. This blog post explores our approach to maintaining a shared understanding of data across our entire stack, from our iOS SDK to our ingest servers, machine learning tools, and beyond.

‍

Introduction to ContextSDK

‍

ContextSDK is designed to empower developers by helping them leverage their users' real-world context. To achieve this, we first collect context signals on the user's iOS device, store them in our data warehouse, build a machine learning model from the data, and ship the new model back in our iOS SDK for the customer to integrate.

‍

Data must flow through a system composed of different components, each developed in the language most suitable for its task. For example, our machine learning code is in Python, our ingest server is a NodeJS application, and our iOS SDK was, of course, written in Swift.

‍

The challenge lies in ensuring that as data traverses through these components, its integrity and structure remain intact, and that each component understands how to read and work with the data.

‍

Centralized Data Definition

‍

At the core of our strategy to achieve data consistency is a centralized tool developed in TypeScript. This tool serves as the single source of truth for all data definitions within ContextSDK. It outlines the signals we track, specifying their data type, name, and unique ID. Whenever a new signal is introduced, it's defined here first, ensuring that every aspect of ContextSDK has a consistent understanding of the data from the get-go.

‍

The beauty of this approach is that it allows us to automatically generate strongly typed bindings for all signals for all our services. Code generation need not be complicated either; we can achieve everything we need by using simple string templating.

‍

We don’t have a fully automatic solution for running DB migrations in our Clickhouse DB yet, but this tool also validates that all defined signals have a matching column in our table. This helps ensure that at release time, all necessary migrations have been written and executed and that the DB is in the shape we expect.

‍

Introducing this tool has really streamlined the process of adding new signals to ContextSDK, as in the past it would have required modifications to the code of multiple services, always with the risk of overlooking one.

‍

Technical Details

‍

This blog is kept intentionally surface level, as the actual code is not very interesting. It essentially boils down to variants of:

‍

‍

For different target applications. For example the output for our Swift codebase will look just like any other Swift enum would:

‍

‍

and for the DB validation we use a simple SQL command

‍

‍

We then compare the output with our listing and print any missing columns, or ones where the data type doesn’t match.

‍

The point here is: Sometimes you shouldn’t get bogged down in trying to find the perfect solution, but simply build something that solves the problem at hand. If it serves the current requirements, and doesn’t take an excessive amount of time to build, it probably is a good investment.

‍

Conclusion

‍

Keeping multiple independent pieces of code in sync doesn’t need to be complicated. We chose to generate code based on some type definitions in TypeScript, but the same could be achieved by sharing a JSON file with all the signals, or anything else, really. The main point is that there should be only a single place where this information, in our case, all the signals we work with, is recorded. By centralizing them, processes around adding and removing signals become clear, and other services can easily be kept in sync with any changes made.

Reinhard Hafenscher
Personalization and Contextualization - Real-World Context in User Tracking
August 23, 2024
Context-Aware Security - Balancing Security and Privacy with User Tracking
August 22, 2024
10 Key Trends Shaping Mobile App Marketing Strategies in 2024
August 21, 2024
AI's Transformative Impact on the Mobile App Industry
August 20, 2024

Subscribe to our newsletter, Contextualize this!

Welcome aboard!

Get ready to explore how real-world context can transform apps. Stay tuned for our upcoming issues!
Oops! Something went wrong while submitting the form.
LoginContact us
Leveraging real‒world user context to supercharge engagement and revenue since 2023.
Founded by Felix Krause and Dieter Rappold.
ContextDecisionContextPushSolutionsProductsDemo App
CompanyContactCareers
Privacy & SecurityPrivacy PolicyImprint
SOC II Type 2GDPR Compliant
© ContextSDK Inc. 169 Madison Avenue, STE 2895 New York, NY 10016 United States
support@contextsdk.com