Update README.md

README.md

@@ -1,2 +1,67 @@
-# code-chunker
-An open-source tool for intelligently chunking and parsing code files, enhancing readability and maintainability by organizing code around key points of interest.
+'''
+# Code Chunker & Parser
+
+The Code Chunker & Parser is a novel open-source tool designed to enhance code readability and maintainability by intelligently chunking code files based on key points of interest. This tool leverages advanced parsing techniques to identify significant elements in your code, such as functions, classes, and comments, to organize your codebase into manageable, easily understandable chunks. It's an invaluable resource for developers looking to optimize their code for better collaboration and efficiency.
+
+## Features
+
+- **Intelligent Chunking:** Break down your code files into chunks around key points of interest like function definitions, class declarations, and crucial comments.
+- **Customizable Token Limits:** Control the size of each chunk with customizable token limits, ensuring that chunks remain manageable and focused.
+- **Support for Multiple Languages:** Initially supporting Python, JavaScript, and CSS, with plans to expand to more programming languages.
+  
+## Getting Started
+
+### Prerequisites
+
+- Python 3.8+
+- OpenAI API key (for token counting features)
+
+### Installation
+
+1. Clone the repository:
+```sh
+git clone https://github.com/yourgithubusername/code-chunker-parser.git
+```
+
+2. Navigate to the project directory
+```sh
+pip install -r requirements.txt
+```
+4. Install the required dependencies
+```sh
+pip install -r requirements.txt
+```
+## Usage
+1. Chunking a Code File:
+Use the CodeChunker class to chunk a specific code file. You can specify the file extension and token limit for chunking.
+Example:
+```py
+from backend.app.util.TextChunker.Chunker import CodeChunker
+
+chunker = CodeChunker(file_extension='py', encoding_name='gpt-4')
+chunks = chunker.chunk(your_code_here, token_limit=1000)
+CodeChunker.print_chunks(chunks)
+```
+2. Parsing Code for Points of Interest:
+
+The CodeParser class allows you to parse code to identify points of interest and comments, which can then be used for chunking or other analysis.
+Example:
+```
+from backend.app.util.CodeParsing.CodeParser import CodeParser
+
+parser = CodeParser(['py'])
+tree = parser.parse_code(your_code_here, 'py')
+points_of_interest = parser.extract_points_of_interest(tree, 'py')
+```
+
+## Contributing
+We welcome contributions from the community, whether it's through reporting bugs, submitting feature requests, or sending pull requests. Please check the CONTRIBUTING.md file for more details on how to contribute to the project.
+
+## License
+This project is licensed under the Apache 2.0 license. See the License file for details
+
+## Acknowledgments
+- This project utilizes the tree-sitter project for parsing code.
+- This also uses tiktoken to count tokens for determining chunk sizes.
+
+