-i}gpdZddlmZddlZddlZddlmZddl m Z ddlm Z ddl m ZGddZdS) z Arabic stop stemmer N)stem_stopwords_const)wordcase)stopword_affixerceZdZdZddZdZdZdZdZdZ d Z e d Z e d Z e d Ze d ZdZe dZe dZe dZe dZdS)StopWordStemmerz Arabic stop stemmer FcLtj|_|jt j|jt jtj|_ |j t j |j t j tj |_d|_t!jd|_i|_i|_||_dS)NTclassedstopwords) tashaphynestemmingArabicLightStemmer comp_stemmerset_prefix_listssconstCOMP_PREFIX_LISTset_suffix_listCOMP_SUFFIX_LIST conj_stemmerCONJ_PREFIX_LISTCONJ_SUFFIX_LIST stop_affixerr generatorallow_syntax_lastmarkstopwordsdictionaryclassStopWordsDictionarystop_dictionarycache_dict_searchcache_affixes_verificationdebugselfrs O/var/www/html/speakWrite/venv/lib/python3.11/site-packages/qalsadi/stem_stop.py__init__zStopWordStemmer.__init__#s&/BBDD ))'*BCCC ))'*BCCC&/BBDD ))'*BCCC ))'*BCCC&688&*" 8K    "$*,' c g}|g}tj|vr@||tjtjdz|D]}|j|}|||tj }|D]}|d|d}||d|d}||dd}|g} |r`| tj r+| |ddtj zn| tj r+| |ddtj znI| tjr*| |ddtjz|tj r#| |tj znA|tjr"| |tjz| D]-}||||||.ѐ|S)z Analyze word morphologically as stop @param stop: the input stop. @type stop: unicode. @return: list of dictionaries of analyzed words with tags. @rtype: list. Nrr)araby ALEF_MADDAappendreplaceALEF_HAMZA_ABOVErsegment verify_affixrCOMP_STOPWORDS_AFFIXESendswithALEF ALEF_MAKSURAYEHTEH TEH_MARBUTA startswithNOONextendsteming_second_level) r!stopdetailed_result stop_list list_seg_compseg procleticstem encletic_nm list_stems r"stemming_stopwordz!StopWordStemmer.stemming_stopword@sf    t # #   T\\%*:Eمدرست+ي = > مدرسة +ي. Return a list of possible cases. @param stem: the input stem. @type stem: unicode. @param suffix_nm: suffix (no mark). @type suffix_nm: unicode. @return: list of stem variants. @rtype: list of unicode. Nr')r\r(r3r7WAWaddr0r2)r@ suffix_nmrm possible_stopvalidated_lists r"r[z!StopWordStemmer.get_stem_variantsHs"!     2I I " I "*   !59,M  " "= 1 1 1 == # # 2 "I(::M  " "= 1 1 1+r$c"tj|}|}|sC|ddtjtjtjfvrtj|rd}dt j|dvrtj|}n|}||fS)u$ Get the suffix variant to be joined to the word. For example: word = مدرس, suffix = ة, encletic = ي. The suffix is converted to Teh. @param word: word found in dictionary. @type word: unicode. @param suffix: second level suffix. @type suffix: unicode. @param enclitic: first level suffix. @type enclitic: unicode. @return: variant of suffixes (vocalized suffix and vocalized suffix without I'rab short mark). @rtype: (unicode, unicode) r'NrL متحركrF) r(strip_tashkeelr2r3r1 is_harakarr`strip_lastharaka)rQrenclitic enclitic_nm newsuffixsuffix_non_irab_marks r"get_suffix_variantsz#StopWordStemmer.get_suffix_variantsks *844   RSS e0%)UZHHH''II 78@H H H#(#9)#D#D #, ...r$ct|tjr6|tjr|ddtjz}|tjtjzr6|tjr|ddtjz}|S)u Get the enclitic variant to be joined to the word. For example: word = عن, suffix = , encletic = ني. The word and enclitic are geminated. @param word: word found in dictionary. @type word: unicode. @param enclitic: first level suffix. @type enclitic: unicode. @return: variant of suffixes (vocalized suffix and vocalized suffix without I'rab short mark). @rtype: (unicode, unicode) rN)r6r(r7r0SHADDAKASRAr3)rQrs r"get_enclitic_variantsz%StopWordStemmer.get_enclitic_variantss$   uz * * 3t}}UZ/H/H 3|el2H   u{UY6 7 7 3DMM%)z0StopWordStemmer.verify_affix..sS   388T&AaD&\4!<,H#I#IZ#W#WA#W#W#Wr$)rQlist_segrs` `r"r.zStopWordStemmer.verify_affix s4         r$ctj|}d|vr |dsdSd|vr |drdSd|vr|drd|vrdSd|vr |dsdSd |vr |dsdSd |vr |d sdSd |vr |drdS|tjkr |dsdSd |vr/|ds%|d tjrdSd|vr |dsdSd|vr |dsdSd|vr |dsdSdS)a Test if the given word from dictionary is compabilbe with affixes tags. @param stop_tuple: the input word attributes given from dictionary. @type stop_tuple: dict. @param affix_tags: a list of tags given by affixes. @type affix_tags:list. @param procletic: first level prefix vocalized. @type procletic: unicode. @param encletic_nm: first level suffix vocalized. @type encletic_nm: unicode. @return: if the tags are compaatible. @rtype: Boolean. r| definitionFdefinedrrGr prepositionrr}pronounu وقايةrQuاستفهامinterroguقسمqasamr~tanwinT)r(rr3r0)rqrtr?rAs r"rbzStopWordStemmer.validate_tagss(33  : % %j.F %5 : % %*Y*? %5 j >* ! J..5 Z   =(A 5 : % %j.H %5  # #Jy,A #5  # # 9(= #5 %) # #J~,F #5 : % % ~ & &*4V*<*E*Eei*P*P &5 z ) )*Z2H )5 z ! !*W*= !5 : % %j.B %5tr$c|S)a Create a list of dictWord objects from dictionary entries @param dict_entries_list: a list of entiers from lexicon @type dict_entries_list: list of dict @return: a list of dictWord object @rtype: a list of dictWord object r)dict_entries_lists r"r^z StopWordStemmer.create_dict_word_s ! r$cFtj||}|S)z ajust vocalization Temporary function @param vocalized: vocalized word. @type vocalized: unicode. @return: ajusted vocalized word. @rtype: unicode. )r AJUSTMENTget)rEajusteds r"rez"StopWordStemmer.ajust_vocalizationjs!#'' 9==r$N)F)__name__ __module__ __qualname____doc__r#rCr9rcrrr staticmethodr[rrrrdr.rbr^rerr$r"rrs:111fHHHT565656n*** +++   \ D!/!/\!/F\4((\(T525252n   \  @@\@D!!\!  \   r$r)rpyarabic.arabyr(tashaphyne.stemmingr tashaphyne.normalizerLrr!arramooz.stopwordsdictionaryclassrralyahmorrrrrr$r"rs------DDDDDD555555 X X X X X X X X X X r$