Private Fine-tuning of Large Language Models with Zeroth-order Optimization

Keyphrases

• Large Language Models 100%
• Zeroth-order Optimization 100%
• Differentially Private Stochastic Gradient Descent 100%
• Scalar 66%
• Memory Efficiency 66%
• Optimization Methods 33%
• Privacy-preserving 33%
• Training Data 33%
• Foundation Models 33%
• Model Size 33%
• Tuning Methods 33%
• High Utility Patterns 33%
• Task Sizes 33%
• Privacy-utility Tradeoff 33%
• Laplace Mechanism 33%

Computer Science

• Gradient Descent 100%
• Large Language Model 100%
• Privacy Preserving 33%
• Training Data 33%

Mathematics

• Stochastics 100%
• Step Size 66%
• Training Data 33%
• Scalar 33%