The Challenge

Scaling and operationalizing presents significant challenges, particularly with maintaining reproducibility and ensuring consistent data pipelines and reliable models, particularly in increasingly regulated fields (governmental agencies, defense contractors, medical research…) . As models move from development to production, simple variations in data sources, transformations, and feature engineering steps can lead to defective models and systems in production and slow down productionisation and development significantly.  

“We Have No Idea How Models will Behave in Production until Production”
How Engineers Operationalize ML - arXiv:2403.16795 - March 2024

Without standardises, and streamlined processes; managing multiple versions of the data and ML assets across environments becomes not only complex but often time the main bottleneck to provide value and solution to organisations, and has Machine Learning Engineers spending valuable time on intricate troubleshooting tasks instead of productive -production ready- deployment of machine learning solutions across the organisation.

Solution Overview

Hopsworks AI Lakehouse enables MLEs to deploy models in production faster with a clear CI/CD process and git support; empowering them with powerful tooling for auditing and versioning across the whole ML lifecycle; from feature engineering to serving endpoints. 

The platform allows for faster experimentation and reuse of both data and engineering logics to reduce the iteration time and facilitate deployment of new models. 

Being framework agnostic, Hopsworks is modular and flexible by nature and is directly usable with the most common tools and technologies across the AI industry; making ML pipelines standardisable and production-ready from day one. 

Built on RonDB (our purpose-built real-time database) and HQS (Hopsworks Query Service), developed internally to power real-time capabilities across the platform, making it the most performant MLOps platform in the world. No tuning required. 

Hopsworks AI Lakehouse is Kubernetes native, making it easier to maintain, deploy, and use scale for any workload, using any compute environment and allowing flexible allocation of resources per projects and deployments, even scaling to zero for model serving.   

Performance Specs

• Feature retrieval: <1ms latency
• Feature freshness: <200ms end-to-end
• Horizontal scaling: Limited only by your K8s cluster

Resources

• ‍Feature Pipelines in Production with Hopsworks: Code, Deployment & Monitoring
  
• RonDB: A Real-Time Database for Real-Time AI Systems
  
• Real-time fraud detection using Feldera and Hopsworks
• From MLOps to ML Systems with Feature/Training/Inference Pipelines
  
• Redefining Feature Stores with Class Leading Performance
  
• Batch & Real-Time ML Systems; A better framework with Hopsworks
  
• Real-Time Machine Learning with Hopsworks

The Challenge

Governance and management of the data for AI, the feature engineering logic and the ML pipelines is essentially the core breaking point of any modern machine learning system in production today. Generating bottlenecks preventing operationalisation of AI in organisation.   

“48% of data engineers spend most of their time resolving data source connections. [...] Half of data engineers say governance takes up more time than anything else” - Unlocking enterprise AI - Nov 2024 - Study by The Economist.

Without a centralized and unified ML ecosystem that empowers data engineering logic to be easily maintainable and data accessible, organizations are struggling in enabling their AI initiatives with their enterprise data, where the core value for personalised and dedicated solution resides. Additionally, most of the legacy data systems are simply not made to support the new AI workloads such as LLMs or real-time use cases, creating a critical value-gap for enterprises and practitioners.

Solution Overview

With the Hopsworks Query Service (HQS) and the world most performant feature store, Hopsworks enables data engineers to get their data from any existing ecosystem; Hopsworks will take care of managing the different cadences, data types and automate the schema management for the models with its powerful abstractions. 

Decoupled Engineering logic between the model and the original data allows practitioners to re-use all the existing data pipelines at any scale for any number of models; all that with queries that can be done in seconds for even the largest of training datasets with integrated query service. 

Transformations can be done on-demand in milliseconds, customized for each model or standardised in the pre-processing of the data; Hopsworks’ flexible and modular approach allows organisation to choose the best frameworks, the best logic, for their use cases without a ramp-up of new bespoke technologies to learn or document. 

Monitoring and validation are natively built-in the platform; making governance and management of the data a painless and automated process; alerting stakeholders when any issue in the pipeline or jobs fails. 

Finally, scalable resources and project management allows for allocation of the computer depending on the needs and requirements of each pipeline. 

Performance Specs

• Query data where it lives - no mandatory data movement
• Automatic schema detection and evolution
• Native support for Delta Lake, Iceberg, and Hudi
• Feature retrieval: <1ms latency
• Feature freshness: <200ms end-to-end

Resources

• Databricks and The Economist; Unlocking Enterprise AI
• RonDB: A Real-Time Database for Real-Time AI Systems
  
• Real-time fraud detection using Feldera and Hopsworks
  
• Live Coding: Real time Features for Real time ML Systems
  
• From MLOps to ML Systems with Feature/Training/Inference Pipelines
  
• Redefining Feature Stores with Class Leading Performance
  
• Feature Pipelines in Production with Hopsworks: Code, Deployment & Monitoring
  
• Batch & Real-Time ML Systems; A better framework with Hopsworks
• Real-Time Machine Learning with Hopsworks

The Challenge

Data scientists spend more time wrestling with data engineering than actual data science. Siloes are responsible for most of the downfall of AI projects in organisation, as data scientists are often isolated in infrastructure and tooling ecosystems that are not consolidated within the rest of the enterprise data environments and are left to build their own data engineering and deployments to solve critical business use cases. Which leads to most of those solutions, never actually being implemented. 

“Everyone at the company should be able to leverage data that drives decisions … without being a data engineer.” - Unlocking enterprise AI - Nov 2024 - Study by The Economist.

The core issue is that the language and technologies used for data science in enterprise are not the same as the ones used in legacy enterprise systems; Python is rarely integrated within the business analytics world and SQL-centric data ecosystems are not made to build and empower implementation and iteration for AI use cases in enterprise.   

Solution Overview

Hopsworks provides an unified ecosystem where data scientists can easily query and engineer new features without being data engineers; a Python-native experience allows data scientists to work with Pandas or Polars and manipulate data frames to build their models and deploy them. 

Transformation logic is decoupled from the original source with powerful abstractions such as the feature view, allowing to re-use features for many models, and new training datasets easily. 

SQL queries, Point in time correct joins, types and schema management is abstracted away by the platform query engine and feature store; data science just have to select the appropriate columns, apply their transformation logic and query for training and inference. 

Hopsworks also removes the barrier from experimentation to production by providing the same APIs for both training and inference; engineering pipelines can be pushed to production and the feature store removes the risks of data leakage or schema miss-management.

Finally, Hopsworks provide an easy Python ecosystem for training or fine-tuning any models, with simple GPU/Compute support.

Example: 

# Get training data
X_train, X_test, y_train, y_test = feature_view.train_test_split(test_size=0.2)
 
# Train model
model = RandomForestClassifier(random_state=42)
model.fit(X_train, y_train)

# Save the model
mr = project.get_model_registry()
model = mr.python.create_model(
    name="some prediction_model",
    metrics={"accuracy": model.score(X_test, y_test)},
    description="some prediction model",
    feature_view = feature_view
)
model.save(model)

# deploy the model
deployment = new_model.deploy(name="new model")
deployment.start()

‍Resources

• Databricks and The Economist: Unlocking Enterprise AI
• From MLOps to ML Systems with Feature/Training/Inference Pipelines
  
• Feature Engineering in Python
  
• Minimal Viable Prediction Service
  
• Hopsworks Query Service: Fast Python API for Lakehouse Data
• Building a Cheque Fraud Detection and Explanation AI System using a fine-tuned LLM
  
• Unlocking the Power of Function Calling with LLMs
  
• Feature Engineering with DBT for Data Warehouses
• ‍Hopsworks real-time, batch and LLM examples

The Challenge

One of the core constraints and limitations to successful implementation of LLM and AI use cases in enterprise is legacy infrastructure. AI has simply moved faster than infrastructure has been able to adapt for the scale and kind of data management required for supporting real-time, constantly evolving and often critical data.  

“As organisations harness AI's potential, they face a critical challenge: their data infrastructure is woefully unprepared [...]. Silos, latency and security are all inhibitors for AI deployment.”
- Unlocking enterprise AI - Nov 2024 - Study by The Economist.

Resource-intensive ML workloads, often developed in multitude of non-compatible environments, deployments with high availability requirements; are simply an infrastructural nightmare and a massive operational cost well; security, compliance and monitoring of those distributed systems is essentially impeding organisations in deploying new products and services empowered with the gold-mine of enterprise data already present.

Solution Overview

Complementing systems of record, Hopsworks provides a unified infrastructure, battle tested at fortune 100s, capable of scaling to support complex workload, and down to reduce operational costs. Not only is it built with reliable technologies such as Kubernetes, it is also, to date, the only high availability online storage for machine learning data in the world, capable of being deployed anywhere. 

Full multi-tenancy control, zero-downtime upgrades and rollbacks, air-gapped if required and hybrid; Hopsworks provides SLAs and monitoring tool that can power any enterprise grade infrastructure for machine learning, while being a complementary system to existing business infrastructure and data layer.

ISO 27001, SOC2 compliant, Hopsworks can also scale horizontally or vertically without downtime. Full monitoring stack and access to log via REST API makes it easy to audit and maintain.

Capabilities you need:

Infrastructure Layer:

• Native K8s deployment with Helm charts
• Automated horizontal/vertical scaling
• Zero-downtime rolling updates
• Air-gapped deployment support
• Multi-region capability

Reliability & Monitoring:

• Prometheus/Grafana integration
• Resource utilization metrics
• Log aggregation via REST API
• Custom alert configurations

Security & Compliance:

• RBAC with LDAP/AD integration
• End-to-end encryption
• Audit logging
• ISO 27001 & SOC2 certified

Resources

• Databricks and The Economist: Unlocking Enterprise AI
• RonDB: A Real-Time Database for Real-Time AI Systems‍
• Redefining Feature Stores with Class Leading Performance‍
• How we secure your data with Hopsworks
• Project Governance in Hopsworks
• Show, don't tell - Hopsworks to Kubernetes
• Hopsworks Public Key Infrastructure: The Unseen Hero
  

The Challenge

With the triple threat of legacy systems, technological stack complexity and governance at large scale, enterprise architects are faced with unprecedented challenges in designing AI-ready infrastructure that can support new workloads while integrating with existing systems that can’t easily be integrated within the new AI technological landscape. A fraction of the organisations globally can support LLMs and Real-time with their current infrastructure; 

“Only 22% of organisations in our survey say their current architecture is fully capable of supporting the unique demands of AI workloads, and just 23% say their current architecture fully integrates AI applications to relevant business data.” - Unlocking enterprise AI - Nov 2024 - Study by The Economist.

Those concerns are only exacerbated by new regulations and the rapid pace of the AI field that demands to generate business value from the current data and deploy more models, faster to stay competitive. 

Solution Overview

Hopsworks AI lakehouse’s flexible and modular architecture integrates easily with the existing data ecosystem and extends the existing data infrastructure to make it capable of supporting complex real-time workloads or new LLM use cases. 

Supporting all major table formats (Delta, Iceberg, Hudi), with native integration for data warehouses and connectors to all existing databases; Hopsworks integrate seamlessly and allows organizations to use their data within any cloud, on-premises, or even hybrid environment all while bringing best-in-class performances to the infrastructure;

• Sub-ms Latency on real-time data retrieval for the most complex use cases 
• Multi-Region High Availability 
• Kubernetes native standards

With all the integration required to support clear governance; end-to-end lineage, role based access control, audit logging and management of data to provide a clear data sovereignty control. 

Resources

• Introducing the AI Lakehouse
• Real-Time Fraud Detection System
• How we secure your data with Hopsworks
• Project governance in Hopsworks
  
• Show, don't tell - Hopsworks to Kubernetes
  
• Hopsworks Public Key Infrastructure: The Unseen Hero
  
• Multi-Region High Availability Comes to Feature Stores