statistics - How to know one system is siginficantly better than another one? -
i studying lexical semantics. have 65 pairs of synonyms sense relatedness. dataset derived paper:
rubenstein, herbert, , john b. goodenough. "contextual correlates of synonymy." communications of acm 8.10 (1965): 627-633.
i extract sentences containing synonyms, transfer neighbouring words appearing in sentences vectors, calculate cosine distance between different vectors, , pearson correlation between distances calculate , sense relatedness given rubenstein , goodenough
i pearson correlation method 1 0.79, , method 2 0.78, example. how measure method 1 better method 2 or not?
well strictly not programming question, since question unanswered in others stackexchange sites, i'll tell approach take.
i there other benchmarks check approaches on similar tasks. can check how method performs on benchmarks , analyze results. methods may capture similarity more while others relatedness , both.
this link wordvec demo automatically scores vectors , provides results.
Comments
Post a Comment