Back to the Index

Online Store

What is an online store?

‍The online store is a row-oriented database or key-value store that provides low-latency lookups for precomputed feature values using one or more entity IDs (or primary keys). The online store enables online inference pipelines (for real-time or near-real-time machine learning systems) to retrieve precomputed features for inference. 

What are the requirements of an online store?

  1. Low latency: One of the primary requirements of an online store is to provide low-latency lookups for features.
  2. Low latency batch lookups:  Personalized search/recommendations at scale are commonly based on a retrieval/ranking architecture that requires looking up in a single batch the features for 100-250 candidates in parallel. Client SLAs typically have a p99 of 50ms for performing the batch lookup.
  3. High availability: The online store should be highly available to ensure that precomputed features can be retrieved at any time without interruptions or downtime. 
  4. High throughput: In addition to low latency, the online store should be able to handle a large number of concurrent requests. It must be capable of serving features to many clients simultaneously without compromising client SLAs.
  5. Scalability: The online store should scale horizontally to handle increasing data volumes and workloads to accommodate the growth of the feature store. This involves being able to add more resources, such as nodes or shards, to the store without downtime or any effects on its performance or availability.
  6. Manageability and monitoring: An online store should be easy to manage and monitor, offering tools and interfaces that allow administrators to monitor performance, identify issues, and manage resources effectively. This helps ensure that the online store operates smoothly and maintains the desired level of performance and availability.

Interested for more?

Does this content look outdated? If you are interested in helping us maintain this, feel free to contact us.
O
AI Lakehouse
AI Pipeline
Auto-regressive Models
AutoML
O
Backfill features
Backfill training data
Backpressure for feature stores
Batch Inference Pipeline
O
CI/CD for MLOps
Compound AI Systems
Context Window for LLMs
O
DAG Processing Model
Data Compatibility
Data Contract
Data Lakehouse
Data Leakage
Data Modeling
Data Partitioning
Data Pipelines
Data Quality
Data Transformation
Data Type (for features)
Data Validation (for features)
Data Warehouse
Data-Centric ML
Dimensional Modeling and Feature Stores
Downstream
Query Engines
O
ELT
ETL
Embedding
Encoding (for Features)
Entity
O
Feature
Feature Data
Feature Engineering
Feature Freshness
Feature Function
Feature Groups
Feature Logic
Feature Monitoring
Feature Pipeline
Feature Platform
Feature Reuse
Feature Selection
Feature Service
Feature Store
Feature Type
Feature Value
Feature Vector
Feature View
Filtering
Fine-Tuning LLMs
O
Generative AI
Gradient Accumulation
Grouped Query Attention
O
Hallucinations in LLMs
Hyperparameter
Hyperparameter Tuning
O
Idempotent Machine Learning Pipelines
In Context Learning (ICL)
Inference Data
Inference Logs
Inference Pipeline
Instruction Datasets for Fine-Tuning LLMs
O
LLM Code Interpreter
LLM Temperature
LLMOps
LLMs - Large Language Models
Lagged features
LangChain
Latent Space
O
ML
ML Artifacts (ML Assets)
MLOps
MLOps Platform
MVPS
Machine Learning Infrastructure
Machine Learning Logs
Machine Learning Observability
Machine Learning Pipeline
Machine Learning Systems
Model Architecture
Model Bias
Model Deployment
Model Development
Model Evaluation (Model Validation)
Model Governance
Model Inference
Model Interpretability
Model Monitoring
Model Performance
O
Natural Language Processing (NLP)
O
Offline Store
On-Demand Features
On-Demand Transformation
Online Inference Pipeline
Online-Offline Feature Skew
Online-Offline Feature Store Consistency
Orchestration
O
KServe
PagedAttention
Pandas UDF
Parameter-Efficient Fine-Tuning (PEFT) of LLMs
Point-in-Time Correct Joins
Precomputed Features
Prompt Engineering
Prompt Store
Prompt Tuning
Python UDF
O
RLHF - Reinforcement Learning from Human Feedback
Real-Time Machine Learning
Recommender System
Representation Learning
Retrieval Augmented Generation (RAG) for LLMs
RoPE Scaling
O
SQL UDF in Python
Sample Packing
Schema
Similarity Search
Skew
Splitting Training Data
Streaming Feature Pipeline
Streaming Inference Pipeline
O
Test Set
Theory-of-Mind Tasks
Time travel (for features)
Train (Training) Set
Training Data
Training Pipeline
Training-Inference Skew
Transformation
Two-Tower Embedding Model
Types of Machine Learning
O
Upstream
O
Validation Set
Vector Database
Versioning (of ML Artifacts)
vLLM