What is Kubeflow?
Kubeflow is an open-source platform for building and running machine learning workflows on Kubernetes. Instead of manually managing training jobs, notebooks, and model deployments across a cluster, Kubeflow gives ML teams a standard way to run the entire ML lifecycle, from experimentation to production, on top of Kubernetes.
The name itself is a mix of “Kube” (Kubernetes) and “Flow” (workflow), positioning it alongside other workflow tools like Airflow and MLflow.
What Kubeflow is used for
- Running Jupyter notebooks: Spin up notebook servers on the cluster for data exploration and experimentation.
- Training models at scale: Distribute model training jobs (TensorFlow, PyTorch, XGBoost, and others) across multiple nodes.
- Building ML pipelines: Chain together steps like data prep, training, and evaluation into a repeatable, automated pipeline (Kubeflow Pipelines).
- Serving models: Deploy trained models as scalable APIs using KServe, Kubeflow’s model-serving component.
- Hyperparameter tuning: Run automated experiments to find the best model configuration.
Each of these runs as a separate component, so teams can use just the pieces they need instead of adopting the whole platform at once.
Is Kubeflow owned by Google?
Not exactly, though it started there. Google created Kubeflow in 2017 as an open-source version of how it ran TensorFlow internally, alongside contributors from Cisco, IBM, Red Hat, and others. Since then, it has moved toward independent, community-led governance: in 2022, Google proposed transferring Kubeflow into the Cloud Native Computing Foundation (CNCF), the same organization that governs Kubernetes itself, and it’s now a CNCF incubating project. So while Google remains a major contributor, Kubeflow isn’t a Google product in the way, say, BigQuery is.
Kubeflow vs MLflow
These two get compared often, but they solve different parts of the same problem:
| Kubeflow | MLflow |
| Runs the entire ML workflow (notebooks, training, pipelines, serving) natively on Kubernetes | Focuses on experiment tracking, model versioning, and a model registry, and can run on Kubernetes or elsewhere |
In practice, many teams use both together: MLflow to track experiments and versions, Kubeflow to actually run the training and serving infrastructure on Kubernetes.
What is Kubeflow Pipelines
Kubeflow Pipelines is the component most teams start with. It lets you define an ML workflow, data preprocessing, training, evaluation, deployment, as a series of connected steps, each running in its own container. Once defined, a pipeline can be rerun, scheduled, or compared across different runs, which makes it easier to reproduce results and track what changed between experiments.
Does Kubeflow has Python SDK
Yes, Kubeflow has a Python SDK, primarily used with Kubeflow Pipelines. It lets you define pipeline steps and their connections directly in Python code, which then gets compiled into a format the pipeline engine can run on the cluster. This is the most common way teams build and manage pipelines, rather than writing raw YAML by hand.
Getting started
Kubeflow runs on any Kubernetes cluster, whether that’s a managed cloud service, on-premises, or even a laptop for testing. The official Kubeflow documentation and its GitHub repository are the standard starting points for setup and examples.
Where it fits in your ML infrastructure
Kubeflow runs on top of the same Kubernetes cluster concepts covered elsewhere in this glossary: it schedules training jobs as pods, relies on the container runtime on each node, and benefits from GPU-backed nodes for training and inference workloads.
