While Large Language Models (LLMs) have shown remarkable potential for automated program repair (APR), their effectiveness is often limited by the availability of high-quality training data. Proprietary industrial codebases are valuable datasets that can be used to enhance model performance but are inaccessible due to data privacy concerns, creating a major challenge for collaborative software development.

Despite that federated learning has emerged as a decentralized privacy-preserving solution, the prevailing paradigm in software engineering research remains centralized, requiring sensitive data to be pooled for model training, which is unacceptable during collaboration. Existing studies overlook challenges in fine-tuning LLMs for generative code-related tasks like program repair, particularly with real-world code heterogeneity in multi-organizational settings. They focus on labeled-data tasks, ignoring feature-skewed generation tasks common in software development, and evaluate only a narrow range of LLM architectures and sizes. Given the value of proprietary codebases, robust privacy is essential in collaborative, distributed development. Moreover, federated approaches to harness collective knowledge while preserving privacy remain largely unexplored.

To address the gap, we investigate federated learning as a privacy-preserving approach for fine-tuning LLMs on proprietary and decentralized data to boost collaborative software development and maintenance. We conduct a comprehensive empirical study to examine the effectiveness of federated learning for program repair to provide practical insights for real-world collaborative software development. Our study makes the following main contributions:

• An empirical study on federated fine-tuning of LLMs for program repair, demonstrating its feasibility and effectiveness of fine-tuning LLMs while {\bf preserving data privacy} in decentralized and collaborative software development.
• Analysis of federated fine-tuning’s impact on the generative code-related task (i.e., program repair), contrasting with prior federated learning work on discriminative tasks and revealing insights for other generative applications.
• Evaluation of a wide range of code LLMs in federated program repair, providing insights into the suitability and practicality of different LLMs in federated learning.
• Investigation of heterogeneous code’s effects on LLM repair capabilities in federated settings, illuminating the robustness and adaptability of LLMs in handling Non-IID data in decentralized environments.
• Assessment of various federated learning algorithms’ impact on LLM-based bug fixing, providing insights into the trade-offs for federated algorithm selection in program repair.