-itdZddlmZmZmZmZddlZ ddlZn #ddl mZYnxYwdZ dZ dZ d Z d Zd ZdS) a- Normalize Utility functions used by to prepare an arabic text to search and index. @author: Taha Zerrouki @author: Taha Zerrouki @contact: taha dot zerrouki at gmail dot com @copyright: Arabtechies, Arabeyes, Taha Zerrouki @license: GPL @date:2017/02/15 @version:0.3 )absolute_importprint_functionunicode_literalsdivisionN)arabyc*tj|S)uStrip vowel from a text and return a result text. The striped marks are : - FATHA, DAMMA, KASRA - SUKUN - SHADDA - FATHATAN, DAMMATAN, KASRATAN, , , . Example: >>> text=u"الْعَرَبِيّةُ" >>> strip_tashkeel(text) العربية @param text: arabic text. @type text: unicode. @return: return a striped text. @rtype: unicode. ) arabconststrip_tashkeeltexts P/var/www/html/speakWrite/venv/lib/python3.11/site-packages/pyarabic/normalize.pyr r s"  #D ) ))c*tj|S)u  Strip tatweel from a text and return a result text. Example: >>> text=u"العـــــربية" >>> strip_tatweel(text) العربية @param text: arabic text. @type text: unicode. @return: return a striped text. @rtype: unicode. )r strip_tatweelr s rrr6s  "4 ( ((rctjtj|}tjtj|S)u"Normalize Hamza forms into one form, and return a result text. The converted letters are : - The converted lettersinto HAMZA are: WAW_HAMZA,YEH_HAMZA - The converted lettersinto ALEF are: ALEF_MADDA, ALEF_HAMZA_ABOVE, ALEF_HAMZA_BELOW ,HAMZA_ABOVE, HAMZA_BELOW Example: >>> text=u"أهؤلاء من أولئكُ" >>> normalize_hamza(text) اهءلاء من اولءكُ @param text: arabic text. @type text: unicode. @return: return a converted text. @rtype: unicode. )r ALEFAT_PATTERNsubALEFHAMZAT_PATTERNHAMZAr s rnormalize_hamzarHs9"  # ' '  = =D  # ' '  > >>rc*tj|S)ueNormalize Lam Alef ligatures into two letters (LAM and ALEF), and return a result text. Some systems present lamAlef ligature as a single letter, this function convert it into two letters, The converted letters into LAM and ALEF are : - LAM_ALEF, LAM_ALEF_HAMZA_ABOVE, LAM_ALEF_HAMZA_BELOW, LAM_ALEF_MADDA_ABOVE Example: >>> text=u"لانها لالئ الاسلام" >>> normalize_lamalef(text) لانها لالئ الاسلام @param text: arabic text. @type text: unicode. @return: return a converted text. @rtype: unicode. )r normalize_ligaturer s rnormalize_lamalefr]s&  ' - --rctjdtjztj|}tjdtjztj|S)uINormalize some spellerrors like, TEH_MARBUTA into HEH,ALEF_MAKSURA into YEH, and return a result text. In some context users omit the difference between TEH_MARBUTA and HEH, and ALEF_MAKSURA and YEh. The conversions are: - TEH_MARBUTA into HEH - ALEF_MAKSURA into YEH Example: >>> text=u"اشترت سلمى دمية وحلوى" >>> normalize_spellerrors(text) اشترت سلمي دميه وحلوي @param text: arabic text. @type text: unicode. @return: return a converted text. @rtype: unicode. z[%s])rerr TEH_MARBUTAHEH ALEF_MAKSURAYEHr s rnormalize_spellerrorsr"rs?( 6'I119=$ G GD 6'I22IM4 H HHrct|}t|}t|}t|}t |}|S)u:Normalize input text and return a result text. Normalize a text by : - strip tashkeel - strip tatweel - normalize Hamza - normalize Lam Alef. - normalize Teh Marbuta and Alef Maksura Example: >>> text=u'أستشتري دمـــى آلية لأبنائك قبل الإغلاق' >>> normalize_searchtext(text) استشتري دمي اليه لابناءك قبل الاغلاق @param text: arabic text. @type text: unicode. @return: return a normalized text. @rtype: unicode. )r rrrr"r s rnormalize_searchtextr$sJ$ $  D   D T " "D 4 D  & &D Kr)__doc__ __future__rrrrrrr r rrrr"r$rrr)s    %%$$$$$$$$***,)))$???*...*III6s#