In recent advancements in machine learning, the ability to reconstruct prompts from the hidden states of large language models (LLMs) has gained attention. This study specifically tests the SIPIT method with two prominent models: GPT-2 and SmolLM.
The findings indicate that SIPIT can effectively recover prompts, although the performance varies between the two models. The analysis includes detailed comparisons of their runtimes, highlighting the uneven performance metrics.
Moreover, the article delves into the implications of these results for the field of prompt engineering, discussing what the recovery means for future applications and research in LLMs.