Synthetic Data Generation for Smarter AI Workflows
IBM Technology explores how synthetic data generation with SDG Hub enables smarter AI workflows.
Open-source AI
A collective of researchers and engineers from Red Hat & IBM building LLM toolkits you can use today.
LLM Hubs
its_hub
Inference-time scaling for LLMs.
sdg_hub
Synthetic data generation pipelines
training_hub
Post training algorithms for LLMs
LLM Tools
async-grpo
Asynchronous GRPO for scalable reinforcement learning.
hopscotch
A method for skipping redundant attention blocks in language models
mini_trainer
Efficient training library for large language models up to 70B parameters on a single node.
orthogonal-subspace-learning
Adaptive SVD-based continual learning method for LLMs.
probabilistic-inference-scaling
Inference-time scaling with particle filtering.
reward_hub
State-of-the-art reward models for preference data generation and acceptance criteria.
SQuat
KV cache quantization for scaling inference time
training
Efficient messages-format SFT library for language models
Recent posts
View all posts
Synthetic data for RAG evaluation: Why your RAG system needs better testing
How SDG Hub enables teams to automatically create grounded evaluation datasets with question-answer-context triplets, transforming RAG tuning from intuition-driven to measurable.
Generate synthetic data for your AI models with SDG Hub
A tutorial on using SDG Hub to turn a small amount of quality data into larger useful datasets through automated pipelines that generate and validate synthetic data.
Recent videos
View all videos📹 Nondeterminism in LLM Inference & Training-Rollout Mismatch
👤 Speaker: Xinheng Ding
📹 Evolutionary Arms Races Between LLMs in Core War
👤 Speaker: Akarsh Kumar
📹 Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
👤 Speaker: Qizheng Zhang