The new features of Hopsworks 2.5 are focused on collaboration and scalability. These features were developed in order to simplify the user experience as well as facilitate data usage and analysis.
Hopsworks 2.5 just landed. This blog post walks through some juicy new features and developments that our customers and open-source users will benefit from.
At Hopworks, everytime we decide to launch a big platform update, we keep in mind four main factors:
Team Collaboration, Data, Deployment, Performance.
In the 2.5 release, new features reflect the team and data factors and were implemented in order to simplify user experience as well as facilitate data usage and assessment.
As a common thread, we will highlight in which category our updates fit.
Improved Git integration was the single most requested feature from our customers and open-source users. We listened to them and Hopsworks 2.5 delivers an integrated experience for data scientists and data engineers with GIt platforms like Github, Gitlab and Bitbucket.
Git integration allows Hopsworks users to clone, view and edit Git repositories directly from the Hopsworks UI. Git integration also enables familiar DevOps practices for feature engineering and MLOps workflows, with the best software engineering practices, such as code reviews and automated testing.
A more in-depth overview of the new Git integration functionality is available in the documentation: https://docs.hopsworks.ai/hopsworks/latest/compute/git/
The ability to quickly look at the business logic that generated a given feature is a key requirement to boost the adoption and reusability of features. The ability to review the code that was used to generate a feature boosts the confidence of data scientists in them, making sure that the feature they are adopting is reliable and provides a useful signal to the model they are building.
The new code preview feature introduced in Hopsworks 2.5 gives data scientists a direct view of the code that was used to build a given feature group. It tracks code changes at the insertion commit level, allowing data scientists and data engineers to review the code and analyze how it has changed over time.
A deep dive into this new feature is available in our previous blog post on how we linked notebooks to features.
Transformation functions in Hopsworks are an additional tool in a data scientist’s toolbox that allows them to apply model-specific logic to features before being used to build and train a model. These transformations can range from feature normalization or one-hot encoding (OHE) to user-defined functions.
Hopsworks has had transformation functions supported for several releases now. In 2.5 we introduced a set of built-in functions that can be used out of the box without the need to write additional code. These functions are: min_max_scaler, standard_scaler, robust_scaler and label_encoder. More details on how to use transformations functions in Hopsworks is available in our documentation.
Hopsworks 2.5 continues with the migration of functionalities and features to a more user-friendly and clear UI. Hopsworks 2.5 contains several new features to control cluster and project settings, UI support for the new features mentioned above as well as many bug fixes and improvements.
The UI is now the default for all users on Hopsworks.ai. Users can still use the old interface if they wish to do so, or if they need to use secondary features that have not yet been migrated.
We have upgraded KubeFlow Model Serving to version 0.6.1 and extended it with support for serving Scikit-Learn models. You can now deploy a model to KubeFlow Model Serving from the UI with a single click or from programs.
There is now also support for serving models behind a REST endpoint managed by ISTIO, giving lower latency predictions compared to making predictions with the Hopsworks REST API. Hopsworks now also includes support to log KFServing prediction requests to Kafka, enabling feature logging from KServing and model monitoring in Hopsworks.
So to summarize all the new features in Hopsworks 2.5:
We introduce how to use the What-If Tool as a Jupyter plugin on Hopsworks to build better machine learning models by making it easier to ask counterfactual questions about your model’s behaviour.