python - How to reduce execution time using thread? -


i have program finding pointwise mutual information of word pairs. code have done follows:

temp_list = co_occurrence_dict.values() n=0 item in temp_list: n += item      main_dict = {}     temp_dict = {}     word in all_opinion_words:         key,value in co_occurrence_dict.iteritems():             if ((word == key.split()[0]) or (word == key.split()[1])):                 pmi_eqn_numerator = (value)/float(n)                 if (key.split()[0] != word):                     temp_word = key.split()[0]                 else:                     temp_word = key.split()[1]                 if temp_word==word:                     temp_dict[temp_word] = 0                 else:                     temp_sum = generate_sum_occurrence_values(temp_word,co_occurrence_dict)                     pmi_eqn_denominator1 = (temp_sum)/float(n)                     temp_sum = generate_sum_occurrence_values(word, co_occurrence_dict)                     pmi_eqn_denominator2 = (temp_sum)/float(n)                     pmi = math.log(pmi_eqn_numerator / (float(pmi_eqn_denominator1) * float(pmi_eqn_denominator2)))                     temp_dict[temp_word] = pmi         temp_list = []         elements in temp_dict.iteritems():             temp_list.append(elements)         main_dict[word] = temp_list  def generate_sum_occurrence_values(item,temp_dict):     sum_values = 0     key,value in temp_dict.iteritems():         if ((item == key.split()[0]) or (item == key.split()[1])):             sum_values += value     return sum_values 

where, all_opinion_words list of words, co_occurrence_dict of form

co_occurrence_dict={'social contemporary': 1,                      'earthly indeed': 1,                     'far mythical': 1,                     'small higher': 1,                      'ideological even': 1,                      'certain `magnificent': 1,                     'back al': 8,                     'perhaps thin': 1,                     'never skeptical': 1,                     'federal small': 1,                     'difficult most': 1,                      'also young': 1,                      'ideological ever': 1,                     'far rather': 1,                     'able happy': 1} 

since have process large number of documents, size of all_opinion_words , co_occurrence_dict large. takes high execution time. read threads can used speed execution. how can apply threads in code?


Comments

Popular posts from this blog

database - VFP Grid + SQL server 2008 - grid not showing correctly -

jquery - Set jPicker field to empty value -

.htaccess - htaccess convert request to clean url and add slash at the end of the url -