![]() ![]() ![]() I thought I had seen a C implementation but couldn’t find it immediately. pdf and breaking it into chunks doesn’t work that well.ģ and 4, virtually everything I’m aware of is in python. There are various pdf data extraction utilities that can be used for 1 and 2 though it’s my view and experience that this is where care needs to be taken to tailor the extraction to the document types and expected query types you have in order to support good answers. Model inference frameworks like llama2.f90 only run the language model for step 5. A language model that can take the extracts plus prompting text and formulate a response. ![]() A retrieval step that takes a query and returns relevant extracts from the database.Indexing in a database, typically as sentence embeddings for semantic search.Ingesting pdf data and formatting appropriately.Longer answer: The way I’ve seen what you’re describing done has several components If such a program doesn’t already exist, I am not ruling out the possibility of creating one in 2024.ĭisclaimer: This post was originally written by a human, but the non-native English has been edited by ChatGPT. I have no prior experience with language models from a programming perspective. I am currently working on machine learning in Fortran, primarily processing economic data. Users would receive pure Fortran code along with comprehensive documentation explaining how to train the model with their own data. I greatly appreciate that the Fortran community offers frontends for their models (although I’m not entirely certain if “frontends” is the correct term), but to be honest, my ideal solution would be a well-documented, user-friendly, and lightweight language model that can be trained by user and implemented in pure Fortran (or alternatively, Rust or Ada). I must admit I have a personal aversion to Microsoft, Google, and especially Facebook. Additionally, I have reservations about foreign companies collecting my data and making moral judgments on acceptable topics. This feature isn’t available for free in ChatGPT, and unfortunately, Google’s Bard is quite buggy, rendering it unusable in my experience. pdf files? For instance, I’d like to place numerous PDFs (such as scientific papers or programming language documentation) into a folder and then pose questions to the language model regarding the content of these files. Can your Fortran Llama2 implementation analyze. I hope my question is neither trivial nor redundant within the documentation. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |