Large Language Models and their Ability to Capture Meaning in Unique Vocabularies

Reserach for my Master’s in Computer Science, Vector Institute Collaborative Specialization in AI
@ Western University, London, Ontario, Canada
Supervisor: Professor
Dan Lizotte
Associate Professor, Department of Computer Science, Westen University


For my Master's research, I am developing interpretability techniques to assess how Large Language Models (LLMS) capture the meaning of words from highly unusual and unique vocabularies. These unique vocabularies include:

LLMs I'm Experimenting With


Experimental Design

I will be developing 3 experiments to discover and evaluate what information LLMs captures when inputting unique vocabulary. Refer to the diagram below to get a high-level overview of how each experiment evaluates different layers of an LLM and the details of each experiment. (All experimental designs shown below were designed and proposed by myself and approved by my professor).
Go Home