How the New Einstein 1 Platform Manages Massive Data and AI Workloads at Scale

In our “Engineering Energizers” Q&A series, we feature Leo Tran, Chief Architect of Platform Engineering at Salesforce. With over 15 years of engineering leadership experience, Leo is instrumental in developing the Einstein 1 Platform. This platform integrates generative AI, data management, CRM capabilities, and trusted systems to provide businesses with personalized customer experiences and AI-driven insights.

Discover how Leo and his team overcame significant technical obstacles and tackled massive scaling challenges to create a powerful platform that revolutionizes the way businesses operate and interact with their customers…

What’s your role in developing the Einstein 1 Platform?

As Chief Architect, my role is to bridge the gap between the various technologies within our layered architecture and how our customers and apps utilize them. Our architecture can be visualized as a layered cake, with each layer serving a specific purpose. The foundation layer comprises a resilient data and infrastructure framework, guaranteeing high availability and data security for our runtimes and data stores.

Above this foundation, the middle layers of the stack offer essential platform capabilities, enabling engineers to evolve existing capabilities, introduce new technologies, and phase out outdated ones, all without disrupting the functionality of our customers’ apps and solutions. This is achieved through the development and upkeep of a unified metadata layer that translates database tables into customer-centric concepts, a security layer for data access control, and an Einstein layer that empowers predictive, generative, and autonomous AI capabilities.

Finally, at the top of the stack, we have the apps that our customers utilize. My role is to optimize this architecture, ensuring that the architecture optimally separates the tough challenges our customers face, allowing them to focus on their core business. Additionally, the architecture supports our internal engineering teams, empowering them to build high-scale, high-availability, and compelling technologies.

A look at Einstein 1 Platform’s layered architecture.

Lastly, my team and I provide essential capabilities for building applications on the Einstein 1 Platform, such as login, data manipulation, app configuration, and event synchronization. These capabilities are unified under Lightning Experience’s capabilities, ensuring a seamless and cohesive Salesforce interface across devices that optimizes the customer experiences.

Leo dives deeper into how he supports Einstein 1 Platform.

What technical issues did the team face when integrating generative AI into the Einstein 1 Platform?

One of the main issues was the need to expand the platform to incorporate new AI technologies without regressing or breaking the existing solutions and apps that were built over the years. The team had to be cautious not to disrupt the tens of thousands of customers relying on the platform.

To mitigate this issue, the team adopted a strategy of staggered rollouts. They made changes incrementally, ensuring thorough testing and using feature gates to control the release of new AI features. The changes were initially rolled out as pilots and betas to a few customers and then gradually expanded to more customers over the course of several months. The team was proud of the fact that some of these significant architectural changes were also successfully rolled out without customers being aware or disrupted.

Leo shares what makes Salesforce Engineering’s culture unique.

What was a key engineering challenge the team faced during the development of the Einstein 1 Platform?

A key challenge was scaling the metadata frameworks and systems to handle a significant increase in the number of schema definitions and use cases. Initially, the Platform supported only 2,000 customer-created entities.

Salesforce supported the creation of customer-created entities by developing an abstraction over a physical database. This introduced the concept of entities and fields to represent data tables and columns, and established the convention of “Salesforce Object Records” (SObject records) to represent rows. This allowed the Platform to handle thousands of entities and work with hundreds of millions of records. This was sufficient for most customers who had hundreds of custom entities capturing data related to their customers and business operations.

However, with the integration of Data Cloud into the Platform and the goal of unifying data from multiple sources, the team needed to seamlessly handle one to two orders of magnitude more metadata. This required scaling the abstraction to accommodate customers working across tens of thousands of entities and billions of SObject records. Additionally, the team had to evolve its conventions for customer data interaction to incorporate new data technologies introduced by Data Cloud, alongside the relational database that Salesforce apps and platform were built on.

Platform data abstraction: Entity represents the schema (metadata) and SObjects represent the data values

How did the Einstein 1 Platform team tackle that significant scaling challenge?

A major architecture evolution was implemented to manage metadata and support the new scale requirements. The complexity of integrating metadata with various platform capabilities, such as data modeling, access controls, CRUD and query APIs, formulas, and UI customizations, needed to be addressed. It was crucial to ensure that any evolution did not disrupt existing functionality or impact customers while maintaining deep integration.

The journey began by evolving how the unique 15-character IDs were defined and assigned to created entities. This involved updating the database tables to accommodate a two orders of magnitude increase in scale.

Next, a carefully planned data migration process was implemented to ensure a smooth transition of entity definitions to the new database tables. This was executed as a staggered rollout, allowing thorough internal testing before gradually deploying it to a few production environments.

Finally, after meticulous testing and refinement, the rollout was successfully completed to all existing customers. The team took great pride in achieving this significant architectural milestone while ensuring minimal impact on all customers.

Moving forward, the team’s focus will be on enhancing the usability of tools and data management APIs to facilitate working with tens of thousands of entities that can span across CRM apps and Data Cloud.

Leo shares why engineers should join Salesforce.

How does your team incorporate customer feedback on AI features to improve Einstein 1 Platform?

To ensure successful adoption and usage of new AI features, the engineering teams employ a comprehensive approach. Internally, various monitoring and metrics frameworks are leveraged to track feature adoption and usage. This includes collecting data on the number of active organizations and users, as well as the frequency of usage per day. By analyzing this data, product assumptions can be validated, customer needs can be identified, and areas for improvement can be pinpointed, all without relying solely on direct customer feedback.

On the customer-facing side, product managers actively engage with customers through focus groups and pilots. Besides gathering feedback about how the feature’s implemented, these interactions also allow for the evaluation of the effectiveness of marketing and customer enablement efforts. The assessment includes determining whether customers are aware of a feature’s existence and if the documentation provides clear instructions. This evaluation helps in determining whether any issues lie in marketing or feature functionality.

In addition to direct customer engagement, Salesforce online communities are leveraged as a valuable feedback channel. The IdeaExchange and Trailblazer Communities provide a platform for customers and partners to share their feedback, propose feature requests, and identify roadmap items. This democratizes the feedback process and fosters a collaborative environment where customers and partners can learn from each other.