Token env

README.md

@@ -18,6 +18,15 @@ Apertus transformer on FastAPI
 
 A FastAPI-based Python application that provides an API to interface with the Apertus LLM from the Swiss AI Initiative.
 
+For more information go to https://huggingface.co/swiss-ai
+
 ## TODOs
 
-- Implement the Apertus Format API https://github.com/swiss-ai/apertus_format
+- [Apertus Format API](https://github.com/swiss-ai/apertus_format)
+- [OpenAI-compatible API](https://medium.com/data-science/how-to-build-an-openai-compatible-api-87c8edea2f06)
+- ...?
+- Profit!
+
+Just kidding, this is an open source project under the [European Public Domain License](LICENSE)
+
+If you have further suggestions please leave them on [Codeberg Issues](https://codeberg.org/loleg/fastapi-apertus/issues)

app.py

@@ -7,6 +7,8 @@ from torch import cuda
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 from huggingface_hub import login
+from dotenv import load_dotenv
+import os
 
 import logging
 # Configure logging
@@ -14,7 +16,10 @@ logging.basicConfig(level=logging.INFO)
 logger = logging.getLogger(__name__)
 
 # Required for access to a gated model
-login()
+load_dotenv()
+hf_token = os.getenv("HUGGING_FACE_TOKEN", None)
+if hf_token is not None:
+    login(token=hf_token)
 
 # Keep data in session
 model = None