Shenzhen Government Similar Question Retrieval Training and Test Sets

Each consultation entry includes information such as the question topic, detailed content, and corresponding responses. In the training set, based on the original crawled questions qi and their respective answers ai, PromptT is utilized to generate semantically positive samples (similar questions) qi+ and hard negative samples (dissimilar questions) qN-, forming the complete triplet data (qi , qi+, qi-) required for contrastive training.

Incorporating answer information provides additional contextual background knowledge for the LLM, simultaneously bridging the semantic gap between different questions sharing identical answers. In the test set, rather than using triplet data, the dataset focuses on generating question pairs (qi , qi') that exhibit stricter semantic equivalence. This approach aims to simulate realistic scenarios encountered in similar question retrieval tasks.