init

README.md

@@ -11,3 +11,39 @@ license: mit
 ---
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+
+
+
+
+1. spaCy 分句， 直接使用 llm（ chatgpt ） 构造合适的query完成这个任务；
+- 使用LLM构造合适的 search query，实验证明比NER强多了
+- 使用citationCount排序搜索结果
+- 稍后会使用rag进行句子/paper相似度匹配，确保不要引用自身paper
+
+2. 从semanticscholar拿到的文章越多越好，先获得query相关文献前30篇，按照citation筛选出前10篇, 这10篇文章是10个json对象，记得向这10个json对象中添加一个key sentence_text
+4. 对intro的每句话重复1-3步，得到N个json对象，名字使用semanticscholar的paperId，对应于intro相关的N个paper，将N篇paper加入加入到本地文献库（大量json文件）
+5. 基于本地本地文献库`papers`，用llamaindex incrementally构建index
+- https://docs.llamaindex.ai/en/stable/examples/discover_llamaindex/document_management/Discord_Thread_Management.html#refresh-the-index-with-new-data
+- docs = papers 要持续更新
+- index 会基于docs增量的更新
+
+6. 构建retriever，https://docs.llamaindex.ai/en/stable/module_guides/querying/retriever/root.html，将每句话作为query从index中搜索语意相似的nodes，从nodes获得bibtex(反推出paperID)，更新intro和.bib
+
+8. 要构建一个自动的metric来评估整个pipeline的准确率
+
+7. **要在每一个semantic query 的结果中去匹配citation**。现在根据sent从所有的index中搜索相似文献，check_node_sentence_match 数值非常低 而且这个方法很慢；解决方案： 
+
+- 不要使用sent去retrieve，而是使用llm生成的query去retrive，另外，好处是本地有一个统一的本地库，随着时间会越来越大; 失败
+
+- 直接使用semantic query 的结果去生成citation，为每一句话构建单独的index，每个index只包含10个文章，用index的retrival对10个文章基于相似度排序 每次任务结束 删除所有本地库；新建一个nb叫做main_lite.ipynb；成功
+
+- retriever.retrieve(sentence) => retriever.retrieve(search_query); 失败
+
+- 尝试不同的 embed model
+
+
+9. gradio 构建demo 
+- https://www.gradio.app/guides/the-interface-class#example-inputs
+
+10. todo: 把所有top 5 都给到用户， 把score写到bib中，让用户根据threshold能自动选择引用论文数量

app.py

@@ -0,0 +1,55 @@
+import gradio as gr
+import time
+import re
+
+from ez_cite import ez_cite
+
+
+
+
+
+example1 = r"""Instead of measuring physical systems and then processing the classical measurement outcomes to infer properties of the physical systems, quantum sensors will eventually be able to transduce quantum information in physical systems directly to a quantum memory, where it can be processed by a quantum computer."""
+
+
+example2 = r"""In order for a learning model to generalise well from training data, it is often crucial to encode some knowledge about the structure of the data into the model itself. Convolutional neural networks are a classic illustration of this principle, whose success at image related tasks is often credited to the existence of model structures that relate to label invariance of the data under translation symmetries. Together with the choice of loss function and hyperparameters, these structures form part of the basic assumptions that a learning model makes about the data, which is commonly referred to as the \textit{inductive bias} of the model.
+
+One of the central challenges facing quantum machine learning is to identify data structures that can be encoded usefully into quantum learning models; in other words, what are the forms of inductive bias that naturally lend themselves to quantum computation? In answering this question, we should be wary of hoping for a one-size-fits-all approach in which quantum models outperform neural network models at generic learning tasks. Rather, effort should be placed in understanding how the Hilbert space structure and probabilistic nature of the theory suggest particular biases for which quantum machine learning may excel. Indeed, an analogous perspective is commonplace in quantum computation, where computational advantages are expected only for specific problems that happen to benefit from the peculiarities of quantum logic."""
+
+example3 = r"""Recurrent neural networks, long short-term memory and gated recurrent neural networks in particular, have been firmly established as state of the art approaches in sequence modeling and transduction problems such as language modeling and machine translation. Numerous efforts have since continued to push the boundaries of recurrent language models and encoder-decoder architectures.
+
+Recurrent models typically factor computation along the symbol positions of the input and output sequences. Aligning the positions to steps in computation time, they generate a sequence of hidden states $h_t$, as a function of the previous hidden state $h_{t-1}$ and the input for position $t$. This inherently sequential nature precludes parallelization within training examples, which becomes critical at longer sequence lengths, as memory constraints limit batching across examples. Recent work has achieved significant improvements in computational efficiency through factorization tricks and conditional computation, while also improving model performance in case of the latter. The fundamental constraint of sequential computation, however, remains. 
+
+Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences. In all but a few cases, however, such attention mechanisms are used in conjunction with a recurrent network. 
+
+In this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs."""
+
+
+
+# wrap the ez_cite.py
+def generate_cite_and_bib_data(introduction):
+    cite_text, bib_data = ez_cite(introduction, debug=False)
+    return cite_text, bib_data
+
+
+
+iface = gr.Interface(
+    fn=generate_cite_and_bib_data,
+    inputs=[gr.Textbox(value='Enter your introduction here', label='introduction', show_label=True, lines=10)],
+    outputs=[gr.Textbox(info='This may take several minutes.', label='.tex', lines=5, show_label=True, show_copy_button=True, interactive=True), 
+             gr.Textbox(info='This may take several minutes.', label='.bib', lines=5, show_label=True, show_copy_button=True, interactive=True)],
+    live=False,
+    title="Easy Cite App (beta-v0.0.5 made by Chip)",
+    description="Enter your introduction and click the buttons to generate Cite Text and Bib Data.",
+    css="""
+        .output { white-space: pre-line; }
+        .container { width: 100%; margin: auto; padding: 5px; }
+        .textbox { width: 100%; }
+        """,
+    examples=[
+        [example1],
+        [example2],
+        [example3]
+    ]
+)
+
+iface.launch(share=True)

ez_cite.py

@@ -0,0 +1,396 @@
+import os
+TOGETHER_API_KEY = 'ed08236bdf2ef77e00cccdbb3d956d958e3db7706cb821b3c7aa269d3faa7620' # os.environ.get("TOGETHER_API_KEY")
+SEMANTIC_SCHOLAR_API_KET = 'JBYSnhdCxL6I1V9zrYUUml8U8jPQmnT1PJLdy7Ia'  # os.environ.get("SEMANTIC_SCHOLAR_API_KET")
+import re
+import sys
+import time
+import json
+import shutil
+import requests
+
+
+import spacy
+#!python -m spacy download en_core_web_lg
+from openai import OpenAI, APIError
+
+import chromadb
+
+
+from llama_index import (
+    VectorStoreIndex,
+    SimpleDirectoryReader,
+    ServiceContext,
+    load_index_from_storage
+)
+from llama_index.embeddings import HuggingFaceEmbedding, TogetherEmbedding
+from llama_index.vector_stores import ChromaVectorStore
+from llama_index.storage.storage_context import StorageContext
+
+
+
+
+
+
+
+
+
+
+
+
+
+# 处理原始.text数据抹去citation，或者直接从用户出获得没有citation的introduct
+def remove_citation(text):
+    # Regular expression to match \cite{...}
+    pattern = r'\\cite\{[^}]*\}'
+
+    # Replace \cite{...} with an empty string
+    text = re.sub(pattern, '', text)
+
+    # Replace multiple spaces with a single space
+    text = re.sub(r' +', ' ', text)
+
+    # Replace spaces before punctuation marks with just the punctuation marks
+    text = re.sub(r"\s+([,.!?;:()\[\]{}])", r"\1", text)
+    
+    return text
+
+
+
+def get_chat_completion(client, prompt, llm_model, max_tokens):
+    messages = [
+        {
+            "role": "system",
+            "content": "You are an AI assistant",
+        },
+        {
+            "role": "user",
+            "content": prompt,
+        }
+    ]
+    try:
+        chat_completion = client.chat.completions.create(
+            messages=messages,
+            model=llm_model,
+            max_tokens=max_tokens
+        )
+        return chat_completion.choices[0].message.content
+    except APIError as e:
+        # Handle specific API errors
+        print(f"API Error: {e}")
+    except Exception as e:
+        # Handle other exceptions
+        print(f"Error: {e}")
+
+
+
+
+
+
+
+
+def get_relevant_papers(search_query, sort=True, count=10):
+    """
+    search_query (str): the required query parameter and its value (in this case, the keyword we want to search for)
+    count (int): the number of relevant papers to return for each query
+
+    Semantic Scholar Rate limit:
+    1 request per second for the following endpoints:
+    /paper/batch
+    /paper/search
+    /recommendations
+    10 requests / second for all other calls
+    """
+    # Define the paper search endpoint URL; All keywords in the search query are matched against the paper’s title and abstract. 
+    url = 'https://api.semanticscholar.org/graph/v1/paper/search'
+    # Define headers with API key
+    headers = {'x-api-key': SEMANTIC_SCHOLAR_API_KET}
+
+    query_params = {
+        'query': search_query,
+        'fields': 'url,title,year,abstract,authors.name,journal,citationStyles,tldr,referenceCount,citationCount',
+        'limit': 20,
+    }
+    # Send the API request
+    response = requests.get(url, params=query_params, headers=headers)
+
+    # Check response status
+    if response.status_code == 200:
+        json_response = response.json() 
+        if json_response['total'] != 0:
+            papers = json_response['data']
+        else:
+            papers = []
+    
+        # Sort the papers based on citationCount in descending order
+        if sort: 
+            papers = sorted(papers, key=lambda x: x['citationCount'], reverse=True)
+      
+        return papers[:count]
+    else:
+        print(f"Request failed with status code {response.status_code}: {response.text}")
+
+
+def save_papers(unique_dir, papers):
+    os.makedirs(unique_dir, exist_ok=True)
+    # Save each dictionary to a separate JSON file
+    for i, dictionary in enumerate(papers):
+        filename = os.path.join(unique_dir, f"{dictionary['paperId']}.json")
+        with open(filename, 'w') as json_file:
+            json.dump(dictionary, json_file, indent=4)
+    print(f"{len(papers)} papers saved as JSON files successfully at {unique_dir}.")
+
+
+
+
+
+
+def get_index(service_context, docs_dir, persist_dir):
+    documents = SimpleDirectoryReader(docs_dir, filename_as_id=True).load_data()
+
+    # check if storage already exists
+    PERSIST_DIR = persist_dir
+    if not os.path.exists(PERSIST_DIR):
+        print('create new index')
+        index = VectorStoreIndex.from_documents(
+            documents, service_context=service_context, show_progress=False
+        )
+        # store it for later
+        index.storage_context.persist(persist_dir=PERSIST_DIR)
+    else:
+        print('load the existing index')
+        # load the existing index
+        storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
+        index = load_index_from_storage(storage_context, service_context=service_context)
+        # refresh the index
+        refreshed_docs = index.refresh_ref_docs(documents, update_kwargs={"delete_kwargs": {"delete_from_docstore": True}})
+        print(f'refreshed_docs:\n{refreshed_docs}')
+
+    return index
+
+
+def get_paper_data(text):
+    """text = node.text """
+    dictionary_from_json = json.loads(text)
+
+    bibtex = dictionary_from_json['citationStyles']['bibtex']
+    bibtex = bibtex.replace('&', 'and')
+    citation_label = re.findall(r'@(\w+){([\w-]+)', bibtex)[0][1]
+    
+    citationCount = dictionary_from_json['citationCount']
+
+    if dictionary_from_json['tldr'] is not None:
+        tldr = dictionary_from_json['tldr']['text']
+    else:
+        tldr = 'No tldr available'
+    
+    url = dictionary_from_json['url']
+    
+    return citation_label, (bibtex, citationCount, tldr, url)
+
+
+def move_cite_inside_sentence(sent, ez_citation):
+    if sent[-1]!='\n':
+        character = sent[-1]
+        sent_new = sent[:-1] + ' <ez_citation>' + character
+    else:  
+        count = sent.count('\n')
+        character = sent[-(count+1)]
+        sent_new = sent[:-(count+1)] + ' <ez_citation>' + character + '\n'*count
+    
+    return sent_new.replace('<ez_citation>', ez_citation)
+
+
+def write_bib_file(bib_file_content, data):
+    bibtex, citationCount, tldr, url = data
+    bib_file_content = bib_file_content + f'\n%citationCount: {citationCount}\n%tldr: {tldr}\n%url: {url}\n' + bibtex
+    return bib_file_content
+    
+
+def write_citation(sent, bib_file_content, retrieved_nodes, sim_threshold=0.75):
+    labels = []
+    for node in retrieved_nodes:
+        citation_label, data = get_paper_data(node.text)
+        print('relevant paper id (node.id_):', node.id_, 'match score (node.score):', node.score)
+        print('relevant paper data:', *data)
+        print('-'*30)
+
+        if node.score > sim_threshold and citation_label != "None":
+            labels.append(citation_label)
+            if not (citation_label in bib_file_content):
+                bib_file_content = write_bib_file(bib_file_content, data)
+        else:
+            continue
+
+    labels = ', '.join(labels)
+    if labels:
+        ez_citation = f'\cite{{{labels}}}'
+        sent_new = move_cite_inside_sentence(sent, ez_citation)
+    else:
+        sent_new = sent
+
+    return sent_new, bib_file_content
+
+
+get_prompt = lambda sentence: f"""
+I want to use semantic scholar paper search api to find the relevant papers, can you read the following text then suggest me an suitable search query for this task? 
+
+Here is an example for using the api:
+<example>
+```python
+import requests
+# Define the paper search endpoint URL
+url = 'https://api.semanticscholar.org/graph/v1/paper/search'
+# Define the required query parameter and its value (in this case, the keyword we want to search for)
+query_params = {{
+    'query': 'semantic scholar platform',
+    'limit': 3
+}}
+# Make the GET request with the URL and query parameters
+searchResponse = requests.get(url, params=query_params)
+```
+</example>
+
+Here is the text:
+<text>
+{sentence}
+</text>
+"""
+
+
+# main block
+def main(sentences, count, client, llm_model, max_tokens, service_context):
+    """count (int): the number of relevant papers to return for each query"""
+    sentences_new = []
+    bib_file_content = ''
+    for sentence in sentences:
+        prompt = get_prompt(sentence)
+        
+        response = get_chat_completion(client, prompt, llm_model, max_tokens)
+    
+        # Define a regular expression pattern to find the value of 'query'
+        pattern = r"'query': '(.*?)'"
+        # Find all matches in the Python code text
+        search_query = re.findall(pattern, response)[0]
+        
+        relevant_papers = get_relevant_papers(search_query, sort=True, count=count)
+        
+        if relevant_papers:
+            # save papers to json files and build index
+            unique_dir = os.path.join("papers", f"{int(time.time())}")
+            persist_dir = os.path.join("index", f"{int(time.time())}")
+            save_papers(unique_dir, relevant_papers)
+            index = get_index(service_context, unique_dir, persist_dir)
+            
+            # get sentence's most similar papers
+            retriever = index.as_retriever(service_context=service_context, similarity_top_k=5)
+            retrieved_nodes = retriever.retrieve(sentence)
+
+            sent_new, bib_file_content = write_citation(sentence, bib_file_content, retrieved_nodes, sim_threshold=0.7)
+            sentences_new.append(sent_new)
+        else:
+            sentences_new.append(sentence)
+        
+        print('sentence:', sentence.strip())
+        print('search_query:', search_query)
+        print('='*30)
+
+    return sentences_new, bib_file_content
+
+
+
+
+def ez_cite(introduction, debug=False):
+    nlp = spacy.load("en_core_web_lg")
+    doc = nlp(introduction)
+    sentences = [sentence.text for sentence in doc.sents]
+    sentences = [ remove_citation(sentence) for sentence in sentences]
+
+    client = OpenAI(api_key=TOGETHER_API_KEY,
+    base_url='https://api.together.xyz',
+    )
+
+    llm_model = "Qwen/Qwen1.5-72B-Chat"
+    max_tokens = 1000
+
+    embed_model = TogetherEmbedding(model_name="togethercomputer/m2-bert-80M-8k-retrieval", api_key=TOGETHER_API_KEY) 
+    service_context = ServiceContext.from_defaults(
+        llm=None, embed_model=embed_model, chunk_size=8192, # chunk_size must be bigger than the whole .json so that all info is preserved, in this case, one doc is one node
+    )
+
+    if debug:
+        sentences = sentences[:2]
+
+    sentences_new, bib_file_content = main(sentences, count=10, 
+                                           client=client,
+                                           llm_model=llm_model, 
+                                           max_tokens=max_tokens, 
+                                           service_context=service_context)
+
+    with open('intro.bib', 'w') as bib_file:
+        bib_file.write(bib_file_content)
+
+    final_intro = ' '.join(sentences_new)
+    print(final_intro)
+    print('='*30)
+
+    dir_path = "index"
+    try:
+        # Delete the directory and its contents
+        shutil.rmtree(dir_path)
+        print(f"Directory '{dir_path}' deleted successfully.")
+    except Exception as e:
+        print(f"Error deleting directory '{dir_path}': {e}")
+
+
+    dir_path = "papers"
+    try:
+        # Delete the directory and its contents
+        shutil.rmtree(dir_path)
+        print(f"Directory '{dir_path}' deleted successfully.")
+    except Exception as e:
+        print(f"Error deleting directory '{dir_path}': {e}")
+
+    return final_intro, bib_file_content
+
+
+
+
+# arXiv:2209.05523v2
+introduction = r"""A long-standing paradigm in machine learning is the trade-off between the complexity of a model family and the model's ability to generalize: more expressive model classes contain better candidates to fit complex trends in data, but are also prone to overfitting noise \cite{nielsen2015neural, geman1992neural}. \textit{Interpolation}, defined for our purposes as choosing a model with zero training error, was hence long considered bad practice  \cite{hastie2009elements}. The success of deep learning - machine learning in a specific regime of extremely complex model families with vast amounts of tunable parameters - seems to contradict this notion; here, consistent evidence shows that among some interpolating models, more complexity tends \textit{not to harm} the generalisation performance, a phenomenon described as "benign overfitting"  \cite{bartlett2021deep}.
+
+In recent years, a surge of theoretical studies have reproduced benign overfitting in simplified settings with the hope of isolating the essential ingredients of the phenomenon \cite{bartlett2021deep, belkin2021fit}. For example, Ref. \cite{bartlett2020benign} showed how interpolating linear models in a high complexity regime (more dimensions than datapoints) could generalize just as well as their lower-complexity counterparts on new data, and analyzed the properties of the data that lead to the "absorption" of noise by the interpolating model without harming the model's predictions. Ref.  \cite{belkin2019reconciling} showed that there are model classes of simple functions that change quickly in the vicinity of the noisy training data, but recover a smooth trend elsewhere in data space (see Figure 1). Such functions have also been used to train nearest neighbor models that perfectly overfit training data while generalizing well, thereby directly linking "spiking models" to benign overfitting \cite{belkin2019does}. Recent works try to recover the basic mechanism of such spiking models using the language of Fourier analysis \cite{muthukumar2020harmless, muthukumar2021classification, dar2021farewell}.
+
+In parallel to these exciting developments in the theory of deep learning, quantum computing researchers have proposed families of parametrised quantum algorithms as model classes for machine learning (e.g. Ref. \cite{benedetti2019parameterized}). These quantum models can be optimised similarly to neural networks \cite{mitarai2018quantum, schuld2019evaluating} and have interesting parallels to kernel methods \cite{schuld2019quantum, havlivcek2019supervised} and generative models  \cite{lloyd2018quantum, dallaire2018quantum}. Although researchers have taken some first steps to study the expressivity  \cite{abbas2021power, wright2020capacity, sim2019expressibility, hubregtsen2021evaluation}, trainability  \cite{mcclean2018barren, cerezo2021cost} and generalisation \cite{caro2021encoding, huang_power_2021, caro2022generalization, banchi2021generalization} of quantum models, we still know relatively little about their behaviour. In particular, the interplay of overparametrisation, interpolation, and generalisation that seems so important for deep learning is yet largely unexplored.
+
+In this paper we develop a simplified framework in which questions of overfitting in quantum machine learning can be investigated. Essentially, we exploit the observation that quantum models can often be described in terms of Fourier series where well-defined components of the quantum circuit influence the selection of Fourier modes and their respective Fourier coefficients \cite{gil2020input, schuld2021effect, wierichs2022general}. We link this description to the analysis of spiking models and benign overfitting by building on prior works analyzing these phenomena using Fourier methods. In this approach, the complexity of a model is related to the number of Fourier modes that its Fourier series representation consists of, and overparametrised model classes have more modes than needed to interpolate the training data (i.e., to have zero training error). After deriving the generalization error for such model classes these "superfluous" modes lead to spiking models, which have large oscillations around the training data while keeping a smooth trend everywhere else. However, large numbers of modes can also harm the recovery of an underlying signal, and we therefore balance this trade-off to produce an explicit example of benign overfitting in a quantum machine learning model.
+
+The mathematical link described above allows us to probe the impact of important design choices for a simplified class of quantum models on this trade-off. For example, we find why a measure of redundancy in the spectrum of the Hamiltonian that defines standard data encoding strategies strongly influences this balance; in fact to an extent that is difficult to counterbalance by other design choices of the circuit.
+
+The remainder of the paper proceeds as follows. We will first review the classical Fourier framework for the study of interpolating models and develop explicit formulae for the error in these models to produce a basic example of benign overfitting (Sec. 2). We will then construct a quantum model with analogous components to the classical model, and demonstrate how each of these components is related to the structure of the corresponding quantum circuit and measurement (Sec. 3). We then analyze specific cases that give rise to "spikiness" and benign overfitting in these quantum models (Sec. 3.2)."""
+
+# # arXiv:2302.01365v3
+# introduction = r"""In order for a learning model to generalise well from training data, it is often crucial to encode some knowledge about the structure of the data into the model itself. Convolutional neural networks are a classic illustration of this principle, whose success at image related tasks is often credited to the existence of model structures that relate to label invariance of the data under translation symmetries. Together with the choice of loss function and hyperparameters, these structures form part of the basic assumptions that a learning model makes about the data, which is commonly referred to as the _inductive bias_ of the model.
+
+# One of the central challenges facing quantum machine learning is to identify data structures that can be encoded usefully into quantum learning models; in other words, what are the forms of inductive bias that naturally lend themselves to quantum computation? In answering this question, we should be wary of hoping for a one-size-fits-all approach in whichquantum models outperform neural network models at generic learning tasks. Rather, effort should be placed in understanding how the Hilbert space structure and probabilistic nature of the theory suggest particular biases for which quantum machine learning may excel. Indeed, an analogous perspective is commonplace in quantum computation, where computational advantages are expected only for specific problems that happen to benefit from the peculiarities of quantum logic.
+
+# In the absence of large quantum computers and in the infancy of quantum machine learning theory, how should we look for insight on this issue? One possibility is to turn to complexity theory, where asymptotic advantages of quantum learning algorithms have been proven. These results are few and far between however, and the enormous gap between what is possible to prove in a complexity-theoretic sense, and the types of advantages that may be possible in practice, means that there are growing doubts about the practical relevance of these results. Indeed, progress in machine learning is often the result of good ideas built on intuition, rather than worst-case complexity theoretic analysis. To repeat a common quip: many problems in machine learning are NP-hard, but neural networks don't know that so they solve them anyway.
+
+# We will take a different route, and lean on the field of quantum foundations to guide us. Quantum foundations is predominantly concerned with understanding the frontier between the quantum and classical world, and typically values a clear qualitative understanding of a phenomenon over purely mathematical knowledge. For these reasons it is well suited to identify features of quantum theory that may advance quantum machine learning in useful directions. In particular, we focus on the phenomenon of contextuality, which is perhaps the most prominent form of nonclassicality studied in the literature. Contextuality has a considerable tradition of being studied in relation to quantum computation, where it is closely connected to the possibility of computational speed-up. Despite this, it has had relatively little attention in quantum machine learning, with only a couple of works linking contextuality to implications for learning.
+
+# We adopt a notion of contextuality called \textit{generalised contextuality}, introduced by Spekkens in 2004. Loosely speaking, it refers to the fact that (i) there are different experimental procedures (called contexts) in the theory that are indistinguishable1, and (ii) any statistical model that reproduces the predictions of the theory must take these contexts into account. With this choice, our first task will then be to introduce a framework to talk about generalised contextuality in machine learning (Section 2). This was missing in previous works, which prove consequences for learning based on phenomena originating from contextuality, but do not attempt to define a notion of contextuality for machine learning that captures a wide range of models. Our general philosophy will be that the framework should depend purely on what a learning model can do, and not on the details of how it does it; i.e., the framework should be independent of the theory on which the models are built. This is necessary to have a framework that treats quantum and classical algorithms on the same footing, and ultimately involves adopting definitions in a similar spirit to the notion of operational contextuality as recently described in.
+
+# We mostly focus on a paradigm of machine learning called multi-task learning, in which the aim is to simultaneously learn a number of separate models for a collection of different (but typically correlated) tasks. Multi-task learning scenarios are conceptually similar to commonly studied contextuality scenarios, and this similarity leads us to a definition of what it means for a multi-task model to be contextual (Section 3). Although the focus on multi-class learning problems appears restrictive, as the separation between tasks is arbitrary at the mathematical level, we also arrive at a notion of contextuality in the single task setting (Section 6). In particular, we argue that it makes sense to think of contextuality as a property relative to a particular inductive bias of a model, rather than a property of the model as a whole.
+
+# Once we have described our framework, our second task will be to identify specific learning problems for which contextuality plays a role (Section 4). We show that this is the case when learning probabilistic models from data sets which feature a linearly conserved quantity in a discrete label space (see Figure 1). Such data sets can arise naturally from experiments involving conserved quantities, zero-sum game scenarios, logistics with conserved resources, substance diffusion in biological systems, and human mobility and migration. We show that the ability of a model to encode the conserved quantity as an inductive bias directly links to a central concept in generalised contextuality, called \textit{operational equivalence}. This results in a constraint on noncontextual learning models that encode the desired bias, which amounts to a limit on the expressivity of noncontextual model classes. For certain data sets, this limitation can negatively impact generalisation performance due to the lack of a suitable model within the class that matches the underlying data distribution; in such cases contextuality may therefore be required for learning. To illustrate this point, in Section 5 we construct a toy problem based on the rock, paper, scissors zero-sum game and prove precise limits on the expressivity of noncontextual model classes that attempt to learn the payoff behaviour of the game.
+
+# In the final part of the work, we study the performance of quantum models for problems that involve our contextuality-inspired bias (Section 7). We first describe two approaches to construct quantum ansatze encoding the bias. The first of these encodes the bias into the state structure of the ansatz, and exploits tools from geometric quantum machine learning. The second approach encodes the bias into the measurement structure, and we present a new family of measurements to this end that may be of independent interest. We then use these tools in a simple numerical investigation (Section 8), inspired by a recent work of Schreiber et al.. Using the fact that quantum machine learning models are equivalent to truncated Fourier series, the authors of define the notion of a classical surrogate model: a linear Fourier features model that has access to the same frequencies of the quantum model, but which lacks its specific inductive bias. The authors found that classical surrogate model classes perform better than quantum model classes on a wide range of regression tasks, the message being that it is still unclear what the inductive bias of quantum machine learning is useful for. In our numerical study, we show that a quantum model class that encodes our contextuality-inspired bias achieves a lower generalisation error than the corresponding surrogate model classes at a specific learning task, even after allowing for regularisation in the surrogate model. We argue that this is due to the fact that the bias cannot be easily encoded into the surrogate model class, which therefore cannot exploit this information during learning.
+
+# In Section 9 we elaborate on a number of areas where contextuality-inspired inductive bias can be expected to play a role in learning. Many of these areas are classical in nature, and therefore suggests that quantum machine learning may be suited to tackling classical learning problems with a specific structure. Finally, in Section 10, we outline our vision for this line of research and the possible next steps to take. Overall, we hope our approach and framework will lead to a new way of thinking about quantum machine learning, and ultimately lead to the identification of impactful problems where the specific structure of quantum theory makes quantum models the machine learning models of choice."""
+
+
+
+
+if __name__ == "__main__":
+
+    final_intro, bib_file_content = ez_cite(introduction, debug=True)
+

requirements.txt

@@ -0,0 +1,236 @@
+aiofiles==23.2.1
+aiohttp==3.9.3
+aiosignal==1.3.1
+albumentations==1.3.1
+altair==5.2.0
+annotated-types==0.6.0
+anthropic-bedrock==0.8.0
+anyio==4.2.0
+appnope @ file:///home/conda/feedstock_root/build_artifacts/appnope_1649077682618/work
+asgiref==3.7.2
+asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1698341106958/work
+async-timeout==4.0.3
+attrs==23.2.0
+backoff==2.2.1
+bcrypt==4.1.2
+blinker==1.7.0
+blis==0.7.11
+boto3==1.34.31
+botocore==1.34.31
+build==1.0.3
+cachetools==5.3.2
+catalogue==2.0.10
+certifi==2023.11.17
+cffi==1.16.0
+charset-normalizer==3.3.2
+chroma-hnswlib==0.7.3
+chromadb==0.4.22
+click==8.1.7
+cloudpathlib==0.16.0
+colorama==0.4.6
+coloredlogs==15.0.1
+comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1704278392174/work
+confection==0.1.4
+contourpy==1.2.0
+cryptography==42.0.2
+cycler==0.12.1
+cymem==2.0.8
+dataclasses-json==0.6.4
+datasets==2.16.1
+debugpy @ file:///private/var/folders/k1/30mswbxs7r1g6zwn8y4fyt500000gp/T/abs_563_nwtkoc/croot/debugpy_1690905063850/work
+decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
+Deprecated==1.2.14
+dill==0.3.7
+dirtyjson==1.0.8
+distro==1.9.0
+en-core-web-lg @ https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.7.1/en_core_web_lg-3.7.1-py3-none-any.whl#sha256=ab70aeb6172cde82508f7739f35ebc9918a3d07debeed637403c8f794ba3d3dc
+exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1704921103267/work
+executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1698579936712/work
+fastapi==0.109.2
+ffmpy==0.3.1
+filelock==3.13.1
+Flask==3.0.2
+flatbuffers==23.5.26
+fonttools==4.48.1
+frozenlist==1.4.1
+fsspec==2023.10.0
+google-auth==2.27.0
+googleapis-common-protos==1.62.0
+gradio==4.18.0
+gradio_client==0.10.0
+greenlet==3.0.3
+grpcio==1.60.1
+h11==0.14.0
+httpcore==1.0.2
+httptools==0.6.1
+httpx==0.26.0
+huggingface-hub==0.20.3
+humanfriendly==10.0
+idna==3.6
+imageio==2.33.1
+importlib-metadata==6.11.0
+importlib-resources==6.1.1
+ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1705418038222/work
+ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1704718870316/work
+itsdangerous==2.1.2
+jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1696326070614/work
+Jinja2==3.1.3
+jmespath==1.0.1
+joblib==1.3.2
+jsonschema==4.21.1
+jsonschema-specifications==2023.12.1
+jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1699283905679/work
+jupyter_core @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_782yoyc_98/croot/jupyter_core_1698937318631/work
+kiwisolver==1.4.5
+kubernetes==29.0.0
+langcodes==3.3.0
+lazy_loader==0.3
+Levenshtein==0.24.0
+lightning==2.1.3
+lightning-utilities==0.10.1
+llama-index==0.9.46
+markdown-it-py==3.0.0
+MarkupSafe==2.1.4
+marshmallow==3.20.2
+matplotlib==3.8.2
+matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
+mdurl==0.1.2
+mmh3==4.1.0
+monotonic==1.6
+mpmath==1.3.0
+multidict==6.0.4
+multiprocess==0.70.15
+munch==4.0.0
+murmurhash==1.0.10
+mypy-extensions==1.0.0
+nest_asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1705850609492/work
+networkx==3.2.1
+nltk==3.8.1
+nougat-ocr==0.1.17
+numpy==1.26.3
+oauthlib==3.2.2
+onnxruntime==1.17.0
+openai==1.12.0
+opencv-python-headless==4.9.0.80
+opentelemetry-api==1.22.0
+opentelemetry-exporter-otlp-proto-common==1.22.0
+opentelemetry-exporter-otlp-proto-grpc==1.22.0
+opentelemetry-instrumentation==0.43b0
+opentelemetry-instrumentation-asgi==0.43b0
+opentelemetry-instrumentation-fastapi==0.43b0
+opentelemetry-proto==1.22.0
+opentelemetry-sdk==1.22.0
+opentelemetry-semantic-conventions==0.43b0
+opentelemetry-util-http==0.43b0
+orjson==3.9.12
+overrides==7.7.0
+packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1696202382185/work
+pandas==2.2.0
+parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
+pdfminer.six==20221105
+pdfplumber==0.10.3
+pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1706113125309/work
+pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
+pillow==10.2.0
+platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1706713388748/work
+posthog==3.4.0
+preshed==3.0.9
+prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1702399386289/work
+protobuf==4.25.2
+psutil @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_1310b568-21f4-4cb0-b0e3-2f3d31e39728k9coaga5/croots/recipe/psutil_1656431280844/work
+ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
+pulsar-client==3.4.0
+pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
+pyarrow==15.0.0
+pyarrow-hotfix==0.6
+pyasn1==0.5.1
+pyasn1-modules==0.3.0
+pycparser==2.21
+pydantic==2.6.0
+pydantic_core==2.16.1
+pydub==0.25.1
+Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1700607939962/work
+PyMuPDF==1.23.20
+PyMuPDFb==1.23.9
+pyparsing==3.1.1
+pypdf==4.0.1
+PyPDF2==3.0.1
+pypdfium2==4.26.0
+PyPika==0.48.9
+pyproject_hooks==1.0.0
+python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
+python-dotenv==1.0.1
+python-Levenshtein==0.24.0
+python-multipart==0.0.9
+pytorch-lightning==2.1.3
+pytz==2023.4
+PyYAML==6.0.1
+pyzmq @ file:///private/var/folders/k1/30mswbxs7r1g6zwn8y4fyt500000gp/T/abs_43pxpbos3z/croot/pyzmq_1705605108344/work
+qudida==0.0.4
+rapidfuzz==3.6.1
+referencing==0.33.0
+regex==2023.12.25
+requests==2.31.0
+requests-oauthlib==1.3.1
+rich==13.7.0
+rpds-py==0.17.1
+rsa==4.9
+ruamel.yaml==0.18.5
+ruamel.yaml.clib==0.2.8
+ruff==0.2.1
+s3transfer==0.10.0
+safetensors==0.4.2
+scikit-image==0.22.0
+scikit-learn==1.4.0
+scipy==1.12.0
+sconf==0.2.5
+semantic-version==2.10.0
+sentencepiece==0.1.99
+shellingham==1.5.4
+six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
+smart-open==6.4.0
+sniffio==1.3.0
+socksio==1.0.0
+spacy==3.7.2
+spacy-legacy==3.0.12
+spacy-loggers==1.0.5
+SQLAlchemy==2.0.25
+srsly==2.4.8
+stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work
+starlette==0.36.3
+sympy==1.12
+tenacity==8.2.3
+thinc==8.2.3
+threadpoolctl==3.2.0
+tifffile==2024.1.30
+tiktoken==0.6.0
+timm==0.5.4
+tokenizers==0.15.1
+tomli==2.0.1
+tomlkit==0.12.0
+toolz==0.12.1
+torch==2.2.0
+torchmetrics==1.3.0.post0
+torchvision==0.17.0
+tornado @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_3a5nrn2jeh/croot/tornado_1696936974091/work
+tqdm==4.66.1
+traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1704212992681/work
+transformers==4.37.2
+typer==0.9.0
+typing-inspect==0.9.0
+typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1702176139754/work
+tzdata==2023.4
+urllib3==2.0.7
+uvicorn==0.27.0.post1
+uvloop==0.19.0
+wasabi==1.1.2
+watchfiles==0.21.0
+wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1704731205417/work
+weasel==0.3.4
+websocket-client==1.7.0
+websockets==11.0.3
+Werkzeug==3.0.1
+wrapt==1.16.0
+xxhash==3.4.1
+yarl==1.9.4
+zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1695255097490/work