arroiw back
Back to Blog

MLOps

Categories

//
5-minute Interviews
//
AI
//
Benchmark
//
Data Engineering
//
Data Science
//
Feature Store
//
SIGMOD 2024

Other Categories

October 21, 2024
12 min
Read

Migrating Hopsworks to Kubernetes

Nearly a year ago, the Hopsworks team embarked on a journey to migrate its infrastructure to Kubernetes. In this article we describe three main pillars of our Kubernetes migration.

Javier Cabrera
September 2, 2024
30 min
Read

Introducing the AI Lakehouse

We describe the capabilities that need to be added to Lakehouse to make it an AI Lakehouse that can support building and operating AI-enabled batch and real-time applications as well LLM applications.

Jim Dowling
August 6, 2024
13 min
Read

Reproducible Data for the AI Lakehouse

We present how Hopsworks leverages its time-travel capabilities for feature groups to support reproducible creation of training datasets using metadata.

Jim Dowling
July 1, 2024
30 min
Read

The Taxonomy for Data Transformations in AI Systems

This article introduces a taxonomy for data transformations in AI applications that is fundamental for any AI system that wants to reuse feature data in more than one model.

Manu Joseph
June 25, 2024
25 min
Read

Modularity and Composability for AI Systems with AI Pipelines and Shared Storage

We present a unified software architecture for batch, real-time, and LLM AI systems that is based on a shared storage layer and a decomposition of machine learning pipelines.

Jim Dowling
April 30, 2024
3 min
Read

Feature Pipelines in Production with Hopsworks

In this post, we will look at how to put feature pipelines into production using Hopsworks.

Fabio Buso
March 21, 2024
19 min
Read

The Enterprise Journey to introducing a Software Factory for AI Systems

In this article we describe the software factory approach to building and maintaining AI systems.

Jim Dowling
March 20, 2024
8 min
Read

GenAI comes to Hopsworks with Vector Similarity Search

Hopsworks has added support for approximate nearest neighbor (ANN) indexing and vector similarity search for vector embeddings stored in its feature store.

Kenneth Mak
February 1, 2024
9 min
Read

F.A.I.R. Principles in Data for AI

At Hopsworks the F.A.I.R principles have been a cornerstone of our approach in designing a platform for managing machine learning data and infrastructure. 

Lex Avstreikh
December 18, 2023
14 min
Read

Multi-Region High Availability Comes to Feature Stores

Learn how Hopsworks achieves high availability for the Online and Offline Feature Store and a Metadata layer, making it an operational ML system both for on-premise and cloud infrastructure.

Antonios Kouzoupis
November 27, 2023
20 min
Read

What is MLOps?

This blog explores MLOps principles, with a focus on versioning, and provides a practical example using Hopsworks for both data and model versioning.

Haziqa Sajid
August 23, 2023
13 min
Read

Automated Feature Engineering with FeatureTools

An ML model’s ability to learn and read data patterns largely depend on feature quality. With frameworks such as FeatureTools ML practitioners can automate the feature engineering process.

Haziqa Sajid
August 11, 2023
20 min
Read

Why Do You Need a Feature Store?

Discover the power of feature stores in modern machine learning systems and how they bridge the gap between model development and production.

Lex Avstreikh
August 9, 2023
13 min
Read

Faster reading from the Lakehouse to Python with DuckDB/ArrowFlight

In this article, we outline how we leveraged ArrowFlight with DuckDB to build a new service that massively improves the performance of Python clients reading from lakehouse data in the Feature Store

Till Döhmen
July 11, 2023
17
Read

MLOps vs. DevOps: Best Practices, Challenges and Differences

Explore the convergence of MLOps and DevOps. Learn about their purposes, differences, and areas of integration and discover best practices, challenges, and their future potential.

Prithivee Ramalingam
June 21, 2023
8 min
Read

Building Feature Pipelines with Apache Flink

Find out how to use Flink to compute real-time features and make them available to online models within seconds using Hopsworks.

Fabio Buso
February 2, 2023
30 min
Read

Feature Store: The missing data layer for Machine Learning pipelines?

In this blog, we discuss the state-of-the-art in data management and machine learning pipelines (within the wider field of MLOps) and present the first open-source feature store, Hopsworks.

Jim Dowling
December 9, 2022
12 min
Read

Optimize your MLOps Workflow with a Feature Store CI/CD and Github Actions

In this blog we present an end to end Git based workflow to test and deploy feature engineering, model training and inference pipelines.

Fabio Buso
September 21, 2022
10 min
Read

Data Validation for Enterprise AI: Using Great Expectations with Hopsworks

Learn more about how Hopsworks stores both data and validation artifacts, enabling easy monitoring on the Feature Group UI page.

Victor Jouffrey
September 7, 2022
15 min
Read

Great Models Require Great MLOps: Using Weights & Biases with Hopsworks

Discover how you can easily make the journey from ML models to putting prediction services in production by choosing best-of-breed technologies.

Moritz Meister
August 23, 2022
20 min
Read

From Pandas to Features to Models to Predictions - A deep dive into the Hopsworks APIs

Learn how the Hopsworks feature store APIs work and what it takes to go from a Pandas DataFrame to features used by models for both training and inference.

Fabio Buso
August 3, 2022
7 min
Read

Introducing the Serverless Feature Store

Hopsworks Serverless is the first serverless feature store for ML, allowing you to manage features and models seamlessly without worrying about scaling, configuration or management of servers.

Jim Dowling
July 26, 2022
9 min
Read

Hopsworks 3.0: The Python-Centric Feature Store

Hopsworks is the first feature store to extend its support from the traditional Big Data platforms to the Pandas-sized data realm, where Python reigns supreme. A new Python API is also provided.

Jim Dowling
April 26, 2022
17 min
Read

Testing feature logic, transformations, and feature pipelines with pytest

Operational machine learning requires the offline and online testing of both features and models. In this article, we show you how to design, build, and run test for features.

Jim Dowling
January 17, 2022
5 min
Read

Model analysis on the What-If framework for TensorFlow on Hopsworks

We introduce how to use the What-If Tool as a Jupyter plugin on Hopsworks to build better machine learning models by making it easier to ask counterfactual questions about your model’s behaviour.

Anastasiia Andriievska
August 3, 2021
12 min
Read

MLOps Wars: Versioned Feature Data with a Lakehouse

With support to Apache Hudi, the Hopsworks Feature Store offers lakehouse capabilities to improve automated feature pipelines and training pipelines (MLOps).

Davit Bzhalava
June 17, 2021
12 min
Read

Hopsworks Online Feature Store: Fast Access to Feature Data for AI Applications

Read about how the Hopsworks Feature Store abstracts away the complexity of a dual database system, unifying feature access for online and batch applications.

Moritz Meister
May 13, 2021
10 min
Read

Distributed ML Experiments on Databricks with Maggy

Learn how to train a ML model in a distributed fashion without reformatting our code on Databricks with Maggy, open source tool available on Hopsworks.

Riccardo Grigoletto
May 12, 2021
4 min
Read

How to manage Python libraries in Hopsworks

This tutorial will show an overview of how to install and manage Python libraries in the platform.

Robin Andersson
April 27, 2021
10 min
Read

Beyond Brainless AI with a Feature Store

Evolve your models from stateless AI to Total Recall AI with the help of a Feature Store.

Jim Dowling
April 27, 2021
7 min
Read

From 100 to ZeRO: PyTorch and DeepSpeed ZeRO on any Spark Cluster with Maggy

Use open-source Maggy to write and debug PyTorch code on your local machine and run the code at scale without changing a single line in your program.

Moritz Meister
February 9, 2021
9 min
Read

How to transform Amazon Redshift data into features with Hopsworks Feature Store

Connect the Hopsworks Feature Store to Amazon Redshift to transform your data into features to train models and make predictions.

Ermias Gebremeskel
January 14, 2021
13 min
Read

Elasticsearch is dead, long live Open Distro for Elasticsearch

In this blog, we describe how we leverage the authentication and authorization support in Open Distro for Elasticsearch to make elasticsearch a project-based multi-tenant service in Hopsworks.

Mahmoud Ismail
October 23, 2020
7 min
Read

Feature Store for MLOps? Feature reuse means JOINs

Use JOINs for feature reuse to save on infrastructure and the number of feature pipelines needed to maintain models in production.

Jim Dowling
October 8, 2020
13 min
Read

ML Engineer Guide: Feature Store vs Data Warehouse

The feature store is a data warehouse of features for machine learning (ML). Architecturally, it differs from the traditional data warehouse in that it is a dual-database.

Jim Dowling
September 30, 2020
16 min
Read

One Function is All you Need: Machine Learning Experiments with Hopsworks

Hopsworks supports machine learning experiments to track and distribute ML for free and with a built-in TensorBoard.

Robin Andersson
June 26, 2020
15 min
Read

Unifying Single-host and Distributed Machine Learning with Maggy

Try out Maggy for hyperparameter optimization or ablation studies now on Hopsworks.ai to access a new way of writing machine learning applications.

Moritz Meister
June 15, 2020
4 min
Read

Manage your own Feature Store on Kubeflow with Hopsworks

Learn how to integrate Kubeflow with Hopsworks and take advantage of its Feature Store and scale-out deep learning capabilities.

Jim Dowling
May 26, 2020
13 min
Read

How to build your own Feature Store

We have many conversations with companies and organizations who are deciding between building their own feature store and buying one. We thought we would share our experience of building one.

Jim Dowling
April 23, 2020
11 min
Read

Hopsworks Feature Store for Databricks

This blog introduces the Hopsworks Feature Store for Databricks, and how it can accelerate and govern your model development and operations on Databricks.

Fabio Buso
April 15, 2020
14 min
Read

ExtremeEarth scales AI to the Earth Observation Community with Hopsworks

Read how ExtremeEarth brings Large-scale AI to the Earth Observation Community with Hopsworks, the Data-intensive AI Platform.

Theofilos Kakantousis
February 14, 2020
15 min
Read

MLOps with a Feature Store

This blog introduces platforms and methods for continuous integration (CI), delivery (CD), and training (CT) with ML platforms, with details on how to do CI/CD MLOps with a Feature Store.

Fabio Buso
November 27, 2019
18 min
Read

Deep Learning for Anti-Money Laundering with a feature store

Deep learning is now the state-of-the-art technique for identifying financial transactions suspected of money laundering. It delivers a lower number of false positives and with higher accuracy.

Jim Dowling