encoding problem!

[impazzito]

Hi to all!

This is my script.
I have some problem with encoding the text...

there are some characters like "è ’ è"

is there e metod to encote it?

Thanks

import appuifw
import globalui
appuifw.app.orientation='automatic'
import re
import urllib

def nascondi():
 try: appuifw.e32.start_exe(u'z:\\sys\\bin\\phone.exe','')
 except: pass

#nascondi()

def ricevi():
 urlWEB = urllib.urlopen
 response = urlWEB('http://oroscopo.libero.it/oro_tab/l.phtml').read()
 try:
  oroscopo = (re.search('<div class="oroscopo_oggi">\x0a(.*?)\x0a</div>', response).group(1))
 except:
  oroscopo = 0
 globalui.global_msg_query(u""+(oroscopo), u"Pesci:")

ricevi()
appuifw.app.set_exit()

[marcelobarrosalmeida]

Hello impazzito

You need to check the encoding your http server is using before converting it to unicode. This issue is a big source of problems and headaches.

My current function to convert from html to unicode is below (decode_html). It is used in wordmobi.

def safe_unicode(value):
    "http://aspyplayer.googlecode.com/svn/trunk/src/aspyplayer.py"
    if type(value) == type(unicode("unicode")):
        return value

    result = ""
    for enc in ['utf8', 'latin1']:
        try:
            result = value.decode(enc)
            break
        except:
            pass
			
    return unicode(result)

def decode_html(line):
    "http://mail.python.org/pipermail/python-list/2006-April/378536.html"
    pat = re.compile(r'&#(\d+);')
    def sub(mo):
        return unichr(int(mo.group(1)))
    return pat.sub(sub, safe_unicode(line))

[impazzito]

Thanks for you reply marcelobarrosalmeida!
in the code of the page, there are character like "& # 2 2 4 ;" (whitout blank spaces)

It is iso-8859-1 or windows-1252 but i don't know how to view it in my python schell!

Can you help me? I have tryed to use ".decompile(iso-8859-1)" but don't work!

Hello impazzito

You need...

[marcelobarrosalmeida]

Hello

Have you tried the code ? I mean:

unicode_text = decode_html(text_from_web)

It will convert the "strange" chars that start with "&#", creating a new unicode string.

[impazzito]

Hello

Have you tried the code ? I mean:

unicode_text = decode_html(text_from_web)

It will convert the "strange" chars that start with "&#", creating a new unicode string.

no.. sorry but i'm very novice in python.. i have to add your code with my code?

[marcelobarrosalmeida]

no.. sorry but i'm very novice in python.. i have to add your code with my code?

Yes ! Just copy the code snippet. Something like (not tested, but should work):

import appuifw
import globalui
import re
import urllib

def safe_unicode(value):
    "http://aspyplayer.googlecode.com/svn/trunk/src/aspyplayer.py"
    if type(value) == type(unicode("unicode")):
        return value

    result = ""
    for enc in ['utf8', 'latin1']:
        try:
            result = value.decode(enc)
            break
        except:
            pass
			
    return unicode(result)

def decode_html(line):
    "http://mail.python.org/pipermail/python-list/2006-April/378536.html"
    pat = re.compile(r'&#(\d+);')
    def sub(mo):
        return unichr(int(mo.group(1)))
    return pat.sub(sub, safe_unicode(line))

def ricevi():
    url = 'http://oroscopo.libero.it/oro_tab/l.phtml'
    response = urllib.urlopen(url).read()
    try:
        oroscopo = (re.search('<div class="oroscopo_oggi">\x0a(.*?)\x0a</div>', response).group(1))
    except:
        oroscopo = u"Impossible to decode"

    msg = decode_html(oroscopo) # <<<<<=========== HERE
    globalui.global_msg_query(msg, u"Pesci:")

appuifw.app.orientation='automatic'
ricevi()
appuifw.app.set_exit()

[impazzito]

WOWOWOWOW!!! Work!!!
May you explain me why i hve to use hxxp://aspyplayer.googlecode.com/svn/trunk/src/aspyplayer.py and hxxp://mail.python.org/pipermail/python-list/2006-April/378536.html in my code? and what will happen if these links will be removed in future?

Thanksssss!!!

Yes ! Just copy the code snippet. Something like (not tested, but should work):

[marcelobarrosalmeida]

WOWOWOWOW!!! Work!!!
May you explain me why i hve to use hxxp://aspyplayer.googlecode.com/svn/trunk/src/aspyplayer.py and hxxp://mail.python.org/pipermail/python-list/2006-April/378536.html in my code? and what will happen if these links will be removed in future?

Thanksssss!!!

Hi,

comments in python starts with "#" so ... you can remove these lines. I decided to preserve them to give credits to original authors.