Projects
The LF AI & Data Foundation supports open source projects within artificial intelligence and the data space.
1chipML
1chipML is an open source library for basic numerical crunching and machine learning for microcontrollers.
Learn MoreAcumos AI
Acumos AI is a platform and open source framework that makes it easy to build, share, and deploy AI apps.
Learn MoreAdlik
Adlik is a toolkit for accelerating deep learning inference. The goal of Adlik is to accelerate deep learning inference process both on cloud and embedded environments.
Learn MoreAdversarial Robustness Toolbox
Adversarial Robustness Toolbox (ART) provides tools that enable developers and researchers to evaluate, defend, certify and verify Machine Learning models and applications against the adversarial threats.
Learn MoreAI Explainability 360
AI Explainability 360 is an open source toolkit that can help users better understand the ways that machine learning models predict labels using a wide variety of techniques throughout the AI application lifecycle.
Learn MoreAI Fairness 360
AI Fairness 360 is an extensible open source toolkit that can help users understand and mitigate bias in machine learning models throughout the AI application lifecycle.
Learn MoreAmundsen
Amundsen is a data discovery and metadata engine for improving the productivity of data analysts, data scientists and engineers when interacting with data.
Learn MoreAngel ML
The Angel Project is a high-performance distributed machine learning platform based on Parameter Server, running on YARN and Apache Spark.
Learn MoreBeyondML
BeyondML is a framework for developing sparse neural networks that can perform multiple tasks across multiple data domains.
Learn MoreBI & AI
The goal of this committee is to integrate the power of AI and BI to make it CI (Cognitive Intelligence) by combing the speed machines accelerate (AI) with the direction intuited by human insight (BI).
Learn MoreBITOL
Within the BITOL project, the primary objective is to tackle multiple challenges, such as data normalization, ensuring the relevance of documentation, establishing service-level expectations, simplifying data and tool integration, and promoting a data product-oriented approach.
Learn MoreCLAIMED
CLAIMED (Component Library for AI, Machine Learning, ETL and Data Science) is a runtime and programming language agnostic Data & AI component framework.
Learn MoreDataOps Committee
The DataOps Committee in LF AI & Data is is a global group that consists of participants from various geographies focused on DataOps.
Learn MoreDataPractices
DataPractices is a “Manifesto for Data Practices,” comprised of values and principles to illustrate the most effective, modern, and ethical approach to data teamwork.
Learn MoreDatashim
Datashim is enabling and accelerating data access for Kubernetes/Openshift workloads in a transparent and declarative way.
Learn MoreDeepCausality
DeepCausality is a hyper-geometric computational causality library that enables fast and deterministic context-aware causal reasoning over complex multi-stage causality models.
Learn MoreDeepRec
DeepRec is a high-performance recommendation deep learning framework based on TensorFlow 1.15, Intel-TensorFlow and NVIDIA-TensorFlow.
Learn MoreDelta
DELTA is a deep learning based end-to-end natural language and speech processing platform.
Learn MoreEgeria
Egeria is the world’s first open source metadata standard. It provides open APIs, event formats, types and integration logic so organizations can share data management and governance across the entireenterprise without reformatting or restricting the data to a single format, platform, or vendor product.
Learn MoreEgeria Conformance
To ensure both consistency and alignment with the standards driven by Egeria, the Egeria Conformance program is available for vendors to showcase how they are shipping Egeria as part of their offering.
Learn MoreElastic Deep Learning
EDL is an Elastic Deep Learning framework designed to help deep learning cloud service providers to build cluster cloud services using deep learning frameworks such as PaddlePaddle and TensorFlow.
Learn MoreElyra
Elyra is an open-source low code / no code framework for creating reproducible, scalable and component based data science pipelines.
Learn MoreFATE
FATE (Federated AI Technology Enabler) is the world's first industrial grade federated learning open source framework to enable enterprises and institutions to collaborate on data while protecting data security and privacy.
Learn MoreFeast
Feast is an open source feature store for machine learning. It was developed as a collaboration between Gojek and Google in 2018.
Learn MoreFlagAI
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale models.
Learn MoreFlyte
Flyte is a production-grade, declarative, structured and highly scalable cloud-native workflow orchestration platform.
Learn MoreForestFlow
ForestFlow is a scalable policy-based cloud-native machine learning model server.
Learn MoreGenerative AI Commons
The LF AI & Data Generative AI Commons is committed to promoting the democratization, advancement, and adoption of efficient, secure, reliable, and ethical Generative AI open source innovations.
Learn MoreHorovod
Horovod makes it easy to take a single-GPU TensorFlow program and successfully train it on many GPUs faster. Horovod also achieved significantly improved GPU resource usage figures.
Learn MoreIntersectional Fairness (ISF)
Intersectional Fairness (ISF) is a bias detection and mitigation technology for addressing intersectional bias, which is caused by the combinations of multiple protected attributes.
Learn MoreJanusGraph
JanusGraph is a scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster.
Learn MoreKedro
Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
Learn MoreKompute
Kompute is a general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous and optimized for advanced GPU data processing use cases.
Learn MoreKServe
KServe provides a Kubernetes Custom Resource Definition for serving machine learning (ML) models on arbitrary frameworks.
Learn MoreLakeSoul
LakeSoul is a cloud-native Lakehouse framework developed by DMetaSoul team, and supports scalable metadata management, ACID transactions, efficient and flexible upsert operation, schema evolution, and unified streaming & batch processing.
Learn MoreLudwig
Ludwig is an open-source, declarative machine learning framework that makes it easy to define deep learning pipelines with a simple and flexible data-driven configuration system.
Learn MoreMachine Learning eXchange
Machine Learning eXchange (MLX) is a Data and AI Assets Catalog and Execution Engine.
Learn MoreMarquez
Marquez is an open source metadata service for the collection, aggregation, and visualization of a data ecosystem’s metadata.
Learn MoreMilvus
Milvus is an open-source vector database that is highly flexible, reliable, and blazing fast.
Learn MoreML Security Committee
The ML Security committee is a global group that advances, showcases and explores challenges and solutions concerning the security of machine learning tooling, systems and use-cases.
Learn MoreMLOps Committee
The LF AI & Data Foundation MLOps Committee helps related projects get more recognization and adoption through cooperation by a passionate community of members.
Learn MoreNNStreamer
NNStreamer is a set of Gstreamer plugins that support ease and efficiency for Gstreamer developers adopting neural network models and neural network developers managing neural network pipelines and their filters.
Learn MoreOpen Platform for Enterprise AI
The mission of the Open Platform for Enterprise AI (OPEA) Project is to develop an ecosystem orchestration framework to efficiently integrate performant GenAI technologies and workflows leading to quicker GenAI adoption and business value.
Learn MoreOpenBytes
OpenBytes aims to facilitate wider sharing of, and collaboration with, data in the AI community through the promotion of data standards and formats and enabling contributions of data.
Learn MoreOpenFL
OpenFL is a Python 3 library for federated learning that enables organizations to collaboratively train a model without sharing sensitive information.
Learn MorePyro
Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend.
Learn MoreRosaeNLG
RosaeNLG is an open source project, template-based Natural Language Generation (NLG) automating the production of relatively repetitive texts based on structured input data and textual templates, run by a NLG engine.
Learn MoreRWKV
Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V).
Learn MoreSapientML
SapientML Project is a Meta-Learning based AutoML initiative designed to enhance the success of the AI model creation process.
Learn MoreShaderNN
ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.
Learn MoreSparklyr
Sparklyr is an open-source and modern interface to scale data science and machine learning workflows using Apache Spark™, R, and a rich extension ecosystem.
Learn MoreSubstra
Substra is a framework offering distributed orchestration of machine learning tasks among partners while guaranteeing secure and trustless traceability of all operations.
Learn MoreThe Open Voice Interoperability Initiative
The Open Voice Network Interoperability Initiative is developing The “Message Envelope,” a universal, open API for voice/chatbot and language model interoperability, analogous to HTTP AND HTML.
Learn MoreThe Open Voice Network Trust Mark Initiative
The Open Voice Network Trust Mark Initiative translates ethical principles into action, focusing on conversational AI.
Learn MoreTrusted AI
The LF AI & Data Trusted AI Committee, a global group working on policies, guidelines, tools and use cases by industry to ensure the development of trustworthy AI systems and processes to develop them continue to improve over time, is now the Responsible AI Workstream of the Generative AI Commons
Learn MoreXtreme1
Xtreme1 is the next generation open source platform for multi-sensory training data.
Learn MoreAdversarial Robustness Toolbox
Adversarial Robustness Toolbox (ART) provides tools that enable developers and researchers to evaluate, defend, certify and verify Machine Learning models and applications against the adversarial threats.
Learn MoreEgeria
Egeria is the world’s first open source metadata standard. It provides open APIs, event formats, types and integration logic so organizations can share data management and governance across the entireenterprise without reformatting or restricting the data to a single format, platform, or vendor product.
Learn MoreFlyte
Flyte is a production-grade, declarative, structured and highly scalable cloud-native workflow orchestration platform.
Learn MoreHorovod
Horovod makes it easy to take a single-GPU TensorFlow program and successfully train it on many GPUs faster. Horovod also achieved significantly improved GPU resource usage figures.
Learn MoreMarquez
Marquez is an open source metadata service for the collection, aggregation, and visualization of a data ecosystem’s metadata.
Learn MoreMilvus
Milvus is an open-source vector database that is highly flexible, reliable, and blazing fast.
Learn MorePyro
Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend.
Learn MoreAdlik
Adlik is a toolkit for accelerating deep learning inference. The goal of Adlik is to accelerate deep learning inference process both on cloud and embedded environments.
Learn MoreAI Explainability 360
AI Explainability 360 is an open source toolkit that can help users better understand the ways that machine learning models predict labels using a wide variety of techniques throughout the AI application lifecycle.
Learn MoreAI Fairness 360
AI Fairness 360 is an extensible open source toolkit that can help users understand and mitigate bias in machine learning models throughout the AI application lifecycle.
Learn MoreAmundsen
Amundsen is a data discovery and metadata engine for improving the productivity of data analysts, data scientists and engineers when interacting with data.
Learn MoreAngel ML
The Angel Project is a high-performance distributed machine learning platform based on Parameter Server, running on YARN and Apache Spark.
Learn MoreDataPractices
DataPractices is a “Manifesto for Data Practices,” comprised of values and principles to illustrate the most effective, modern, and ethical approach to data teamwork.
Learn MoreDatashim
Datashim is enabling and accelerating data access for Kubernetes/Openshift workloads in a transparent and declarative way.
Learn MoreDelta
DELTA is a deep learning based end-to-end natural language and speech processing platform.
Learn MoreElastic Deep Learning
EDL is an Elastic Deep Learning framework designed to help deep learning cloud service providers to build cluster cloud services using deep learning frameworks such as PaddlePaddle and TensorFlow.
Learn MoreFATE
FATE (Federated AI Technology Enabler) is the world's first industrial grade federated learning open source framework to enable enterprises and institutions to collaborate on data while protecting data security and privacy.
Learn MoreFeast
Feast is an open source feature store for machine learning. It was developed as a collaboration between Gojek and Google in 2018.
Learn MoreForestFlow
ForestFlow is a scalable policy-based cloud-native machine learning model server.
Learn MoreJanusGraph
JanusGraph is a scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster.
Learn MoreKedro
Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
Learn MoreKompute
Kompute is a general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous and optimized for advanced GPU data processing use cases.
Learn MoreKServe
KServe provides a Kubernetes Custom Resource Definition for serving machine learning (ML) models on arbitrary frameworks.
Learn MoreLudwig
Ludwig is an open-source, declarative machine learning framework that makes it easy to define deep learning pipelines with a simple and flexible data-driven configuration system.
Learn MoreNNStreamer
NNStreamer is a set of Gstreamer plugins that support ease and efficiency for Gstreamer developers adopting neural network models and neural network developers managing neural network pipelines and their filters.
Learn MoreRWKV
Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V).
Learn MoreSparklyr
Sparklyr is an open-source and modern interface to scale data science and machine learning workflows using Apache Spark™, R, and a rich extension ecosystem.
Learn MoreSubstra
Substra is a framework offering distributed orchestration of machine learning tasks among partners while guaranteeing secure and trustless traceability of all operations.
Learn MoreThe Open Voice Interoperability Initiative
The Open Voice Network Interoperability Initiative is developing The “Message Envelope,” a universal, open API for voice/chatbot and language model interoperability, analogous to HTTP AND HTML.
Learn MoreThe Open Voice Network Trust Mark Initiative
The Open Voice Network Trust Mark Initiative translates ethical principles into action, focusing on conversational AI.
Learn More1chipML
1chipML is an open source library for basic numerical crunching and machine learning for microcontrollers.
Learn MoreBeyondML
BeyondML is a framework for developing sparse neural networks that can perform multiple tasks across multiple data domains.
Learn MoreBITOL
Within the BITOL project, the primary objective is to tackle multiple challenges, such as data normalization, ensuring the relevance of documentation, establishing service-level expectations, simplifying data and tool integration, and promoting a data product-oriented approach.
Learn MoreCLAIMED
CLAIMED (Component Library for AI, Machine Learning, ETL and Data Science) is a runtime and programming language agnostic Data & AI component framework.
Learn MoreDeepCausality
DeepCausality is a hyper-geometric computational causality library that enables fast and deterministic context-aware causal reasoning over complex multi-stage causality models.
Learn MoreDeepRec
DeepRec is a high-performance recommendation deep learning framework based on TensorFlow 1.15, Intel-TensorFlow and NVIDIA-TensorFlow.
Learn MoreElyra
Elyra is an open-source low code / no code framework for creating reproducible, scalable and component based data science pipelines.
Learn MoreFlagAI
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale models.
Learn MoreIntersectional Fairness (ISF)
Intersectional Fairness (ISF) is a bias detection and mitigation technology for addressing intersectional bias, which is caused by the combinations of multiple protected attributes.
Learn MoreLakeSoul
LakeSoul is a cloud-native Lakehouse framework developed by DMetaSoul team, and supports scalable metadata management, ACID transactions, efficient and flexible upsert operation, schema evolution, and unified streaming & batch processing.
Learn MoreMachine Learning eXchange
Machine Learning eXchange (MLX) is a Data and AI Assets Catalog and Execution Engine.
Learn MoreOpen Platform for Enterprise AI
The mission of the Open Platform for Enterprise AI (OPEA) Project is to develop an ecosystem orchestration framework to efficiently integrate performant GenAI technologies and workflows leading to quicker GenAI adoption and business value.
Learn MoreOpenBytes
OpenBytes aims to facilitate wider sharing of, and collaboration with, data in the AI community through the promotion of data standards and formats and enabling contributions of data.
Learn MoreOpenFL
OpenFL is a Python 3 library for federated learning that enables organizations to collaboratively train a model without sharing sensitive information.
Learn MoreRosaeNLG
RosaeNLG is an open source project, template-based Natural Language Generation (NLG) automating the production of relatively repetitive texts based on structured input data and textual templates, run by a NLG engine.
Learn MoreSapientML
SapientML Project is a Meta-Learning based AutoML initiative designed to enhance the success of the AI model creation process.
Learn MoreShaderNN
ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.
Learn MoreXtreme1
Xtreme1 is the next generation open source platform for multi-sensory training data.
Learn MoreAcumos AI
Acumos AI is a platform and open source framework that makes it easy to build, share, and deploy AI apps.
Learn MoreTrusted AI
The LF AI & Data Trusted AI Committee, a global group working on policies, guidelines, tools and use cases by industry to ensure the development of trustworthy AI systems and processes to develop them continue to improve over time, is now the Responsible AI Workstream of the Generative AI Commons
Learn MoreBI & AI
The goal of this committee is to integrate the power of AI and BI to make it CI (Cognitive Intelligence) by combing the speed machines accelerate (AI) with the direction intuited by human insight (BI).
Learn MoreDataOps Committee
The DataOps Committee in LF AI & Data is is a global group that consists of participants from various geographies focused on DataOps.
Learn MoreGenerative AI Commons
The LF AI & Data Generative AI Commons is committed to promoting the democratization, advancement, and adoption of efficient, secure, reliable, and ethical Generative AI open source innovations.
Learn MoreML Security Committee
The ML Security committee is a global group that advances, showcases and explores challenges and solutions concerning the security of machine learning tooling, systems and use-cases.
Learn MoreMLOps Committee
The LF AI & Data Foundation MLOps Committee helps related projects get more recognization and adoption through cooperation by a passionate community of members.
Learn More