Training From Scratch vs Pre-trained Models

Developers should use training from scratch when working with highly specialized or novel datasets where pre-trained models are unavailable or ineffective, such as in niche scientific research or custom industrial applications meets developers should learn and use pre-trained models when building ai applications with limited data, time, or computational power, as they provide a strong starting point that can be customized for specific needs. Here's our take.

Training From Scratch

Nice Pick

Developers should use training from scratch when working with highly specialized or novel datasets where pre-trained models are unavailable or ineffective, such as in niche scientific research or custom industrial applications

Pros

• It is also appropriate when computational resources are sufficient and the goal is to avoid biases or limitations from pre-trained models, ensuring the model is tailored specifically to the task at hand
• Related to: machine-learning, deep-learning

Cons

• Specific tradeoffs depend on your use case

Pre-trained Models

Developers should learn and use pre-trained models when building AI applications with limited data, time, or computational power, as they provide a strong starting point that can be customized for specific needs

Pros

• They are essential in domains like NLP for tasks such as sentiment analysis or chatbots using models like BERT, and in computer vision for object detection or image classification using models like ResNet
• Related to: transfer-learning, machine-learning

Cons

• Specific tradeoffs depend on your use case

The Verdict

These tools serve different purposes. Training From Scratch is a methodology while Pre-trained Models is a concept. We picked Training From Scratch based on overall popularity, but your choice depends on what you're building.

The Bottom Line

Training From Scratch wins

Based on overall popularity. Training From Scratch is more widely used, but Pre-trained Models excels in its own space.