“结巴”中文分词:做最好的 Python 中文分词组件
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

21 lines
403 B

from __future__ import print_function
import sys
import time
sys.path.append("../")
import jieba
jieba.initialize()
import jieba.posseg as pseg
url = sys.argv[1]
content = open(url,"rb").read()
t1 = time.time()
words = list(pseg.cut(content))
t2 = time.time()
tm_cost = t2-t1
log_f = open("1.log","w")
log_f.write(' / '.join(map(str, words)))
print('speed' , len(content)/tm_cost, " bytes/second")