Hugging Face, Inc 🤗 (huggingface.co) is a New York City based French-American company that develops tools for building applications using machine learning. It is a company known for its work in the field of AI, particularly in natural language processing (NLP). Founded in 2016, Hugging Face has become notable for its open-source contributions and the development of Transformer-based models that have significantly advanced the capabilities of NLP applications.
Key Aspects of Hugging Face
- Transformer Models: Hugging Face is most famous for its library called 'Transformers'. This library provides thousands of pre-trained models to perform tasks on texts such as classification, information extraction, question answering, summarization, translation, and text generation. These models include BERT, GPT, T5, and DistilBERT, among others.
- Open-Source Approach: One of the reasons for Hugging Face's popularity is its commitment to open-source. The company actively maintains and updates its libraries, making them accessible to researchers, developers, and businesses.
- Community and Collaboration: Hugging Face has fostered a strong community around its tools, encouraging collaboration, sharing of models, and discussion on best practices in NLP. This has made it a go-to resource for many working in AI and NLP.
- Easy Integration: The Transformers library is designed for ease of use, allowing for the integration of state-of-the-art NLP models with minimal effort. It supports several programming languages, primarily Python, and can be integrated with other machine learning frameworks like TensorFlow and PyTorch.
- Hugging Face Hub: This is a platform where the community can share and collaborate on models. Users can upload their own models, download others, and contribute to the ongoing improvement of NLP technologies.
- Commercial Services: Beyond its open-source contributions, Hugging Face also offers commercial services. This includes enterprise solutions for companies looking to leverage advanced NLP models in their products and services.
- Research Contributions: Hugging Face is actively involved in AI research, often publishing papers and contributing to the advancement of NLP technologies. They collaborate with academic institutions and other research organizations.
- Tokenizers and Datasets: Alongside the Transformers library, Hugging Face also develops and maintains libraries for efficient tokenization (necessary for preparing text for processing by models) and a vast collection of datasets, which are crucial for training and evaluating NLP models.
Hugging Face has become a pivotal player in the AI field, especially in democratizing access to cutting-edge NLP technologies. Their tools are used by a wide range of users, from independent developers to large corporations, making significant contributions to the advancement and accessibility of NLP.
Here are some helpful articles:
- RLHF (conceptual overview): 
- RLHF with Meta's Llama models (conceptual + code): 
- Reward modelling (code): 
- Quantisation deep-dive: 
- Quantisation in practice: 
- Qlora: 
Models being used:
- LLM for fine-tuning: 
- LLM for reward modelling (or unstructured regression modelling / URM as we can refer to it):  ( is another contender here)
- Reinforcement Learning from Human Feedback: From Zero to chatGPT - One hour 🤗 lecture... you can skip the first 5 minutes and the most important bit is only 35 mins, then Q&A.
- Machine Learning - Has a glossary.