- Multi-HPC Orchestration: Enables large-scale pre-training and fine-tuning across one or more supercomputers using specialized distributed frameworks.
- Simplified Interface: Leverages high-level abstraction layers that reduce the need for direct interaction with complex SLURM scheduling commands.
- Privacy-Preserving Workflows: Facilitates collaborative model development by keeping sensitive data localized at the source while aggregating global insights.
Key Value: Accelerating model development on geographically distributed datasets while maintaining strict data sovereignty and reducing infrastructure complexity.
Practical example
Organizations often need to train models on data stored across different jurisdictions or computing centers.
This service facilitates, among other activities:
- Cross-Center Training: Adapting pre-existing Large Language Models (LLMs) using data distributed across multiple EuroHPC-JU computing centers.
- Template-Driven Setup: Utilizing specialized xFFL templates tailored to specific use cases to rapidly deploy federated workflows.
- No-Move Data Analysis: Training robust, domain-specific chatbots by accessing local data repositories directly, eliminating the cost and risk of massive data transfers.
Case Study: The service supports a European consortium in training a specialized legal assistant using data hosted in three separate countries. By deploying an xFFL template, high model accuracy is achieved without moving any byte of sensitive data across borders.
