changes for python

README.md

@@ -1,12 +1,151 @@
 ---
 title: MT564AITraining
-emoji: ⚡
+emoji: 🚀
 colorFrom: blue
 colorTo: gray
 sdk: docker
 pinned: false
 license: apache-2.0
 short_description: MT564Model training
+
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+# SWIFT-MT564-Assistant
+  Added MT564 TinyLlama training interface
+✓ Created comprehensive training UI with file upload
+✓ Integrated horoscope harvesting with MT564 training
+✓ Both systems running in unified application
+✓ Navigation links connect both functionalities
+
+The application now provides both data harvesting for horoscopes AND MT564 TinyLlama training with a complete UI. You can access the MT564 training interface through the navigation menu.
+
+## Project Overview
+
+This project creates an AI-powered documentation assistant for financial messaging standards, specifically focused on the SWIFT MT564 message type. It combines web scraping, data processing, TinyLlama fine-tuning, and a user-friendly interface to provide an intelligent assistant for financial messaging professionals.
+
+## Key Components
+
+### 1. Data Collection & Processing
+
+- **Web Scraper**: Extracts structured data from [ISO20022 SWIFT MT564 documentation](https://www.iso20022.org/15022/uhb/finmt564.htm)
+- **PDF Parser**: Extracts text and structural information from uploaded SWIFT documentation PDFs
+- **Data Formatter**: Converts scraped and parsed data into training examples for the model
+
+### 2. Model Training Pipeline
+
+- **TinyLlama Integration**: Implementation of TinyLlama, a smaller and more efficient LLM
+- **Fine-tuning Scripts**: Specialized scripts for training on SWIFT message documentation
+- **Evaluation Tools**: Methods to test the model's understanding of SWIFT message formats
+
+### 3. User Interface
+
+- **Web Application**: Flask-based interface for interacting with the model
+- **PDF Upload**: Functionality to upload and process SWIFT documentation PDFs
+- **Question-Answering System**: Interactive chat interface for asking questions about MT564 and related formats
+
+## Technical Architecture
+
+```
+SWIFT-MT564-Assistant/
+├── scrapers/                 # Web scraping components
+│   ├── iso20022_scraper.py   # Scraper for ISO20022 website
+│   ├── pdf_parser.py         # PDF extraction utilities
+│   └── data_processor.py     # Converts raw data to training format
+│
+├── model/                    # ML model components
+│   ├── tinyllama_trainer.py  # Fine-tuning implementation
+│   ├── data_formatter.py     # Prepares data for training
+│   └── evaluator.py          # Tests model performance
+│
+├── webapp/                   # Web application
+│   ├── app.py                # Flask application
+│   ├── templates/            # HTML templates
+│   │   ├── index.html        # Main page
+│   │   └── result.html       # Results display
+│   └── static/               # CSS, JS, and other static files
+│
+├── data/                     # Data storage
+│   ├── raw/                  # Raw scraped data
+│   ├── processed/            # Processed training data
+│   └── uploaded/             # User-uploaded PDFs
+│
+├── train_mt564_model.py      # Script to train the model
+├── requirements.txt          # Project dependencies
+└── README.md                 # Project documentation
+```
+
+## How It Works
+
+1. **Data Collection Phase**:
+   - The ISO20022 scraper extracts structured data from the SWIFT MT564 documentation
+   - The data is processed and converted into a training dataset of instruction-response pairs
+
+2. **Model Training Phase**:
+   - TinyLlama is fine-tuned on the specialized SWIFT message format data
+   - The model learns the structure, fields, and usage of MT564 messages
+
+3. **User Interaction Phase**:
+   - Users upload SWIFT documentation PDFs through the web interface
+   - The system extracts and processes the PDF content
+   - Users ask questions about SWIFT messages and receive accurate, contextual responses
+
+## Installation & Setup
+
+### Prerequisites
+
+- Python 3.8+
+- PyTorch
+- Transformers library
+- Flask
+- PDF processing libraries
+
+### Installation Steps
+
+```bash
+# Clone the repository
+git clone <repository-url>
+cd SWIFT-MT564-Assistant
+
+# Create a virtual environment
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+
+# Install dependencies
+pip install -r requirements.txt
+
+# Download and prepare the model
+python prepare_mt564_data.py
+
+# Run the web application
+python main.py
+```
+
+## Usage
+
+### Training the Model
+
+```bash
+# Run the scraper to collect data
+python scrapers/iso20022_scraper.py
+
+# Process the data
+python scrapers/data_processor.py
+
+# Train the model
+python train_mt564_model.py
+```
+
+### Using the Web Interface
+
+1. Start the Flask application: `python main.py`
+2. Open a browser and navigate to: `http://localhost:5000`
+3. Upload a SWIFT MT564 documentation PDF
+4. Ask questions about the SWIFT message format
+
+## Future Enhancements
+
+- Expand coverage to additional SWIFT message types (MT565, MT566, etc.)
+- Implement multi-document reasoning across different SWIFT standards
+- Add support for ISO20022 MX message formats
+- Develop specialized modules for message validation and conversion