Sending Json from phone to server

[rager]

Hi, I have a form which will try to send data to a server side php script. However, whenever the send_data function is executed, the phone cannot send the data over. I inserted a few print statements and I found that it is stuck at the "print json.write(review)" part. I notice that it didn't even ask me for which access point to connect which I assume that the conn is not even establised properly. I tried not using Json and that works. Any idea how I can make Json work?

    def send_data(reviewData):
        appuifw.app.title = u'Saving Data'
        chunk = None
        try:
            print 'you opening files'
            f= open(PHOTO,'rb')
            chunk = f.read()
            f.close()
        except:
            print 'There are no photos'
        print 'start forming msg'
        review = {'photo':chunk ,\
               'title':reviewData[0][2],\
               'shop':reviewData[1][2],\
               'rating':reviewData[2][2][0][reviewData[2][2][1]], \
               'comment':reviewData[3][2]}
        headers = {'Content-type':'application/x-www-form-urlencoded',
                   'Accept':'text/plain'}
        print json.write(review)
        conn = httplib.HTTPConnection("testingserver.com")
        conn.request("POST", "/uploadReview.php", urllib.urlencode(json.write(review)) , headers)
        appuifw.app.title = u'Data saved'

[jethro.fn]

PyS60 and Python do not include a JSON translator by default. Which third party translator are you using?

[rager]

I am using the one provided at www.mobilepythonbook.com.
I am certain that the json works, because I have tried other examples from the book. I just cannot understand why it does not work in this case.

[jethro.fn]

Print statements only make sense when running the PyS60 program inside Script Shell or PED or similar. Standalone applications cannot use print statements without extra preparation.

Also, you're dumping the bytes of a JPEG image to screen (assuming PHOTO points to what I think it does). I don't think the Text() control likes that much. I tried your code snippet on a a PC and it messed up the console pretty bad.

[rager]

My bad, i forgot to state that this code is meant for the phone. I have installed a python interpreter on my phone ver 1.4.2 and the code is ran in the phone. The print statments are just for debugging purpose. Assuming that I remove the print statments, it will still not work. Any ideas why?

[gaba88]

hi rager
did you tried the exact code. and what sort of error you got when implementing this application.
give a feedback

[jethro.fn]

My bad, i forgot to state that this code is meant for the phone. I have installed a python interpreter on my phone ver 1.4.2 and the code is ran in the phone. The print statments are just for debugging purpose.

Yeah, I figured that much. That's why I referred to the PyS60 Text() control above. It is responsible for displaying text on-screen, and your print statement flooded it with binary data. This is bound to result in all kinds of weird behaviour.

Assuming that I remove the print statments, it will still not work. Any ideas why?

You're passing a string to urllib.urlencode(). It only takes a mapping or a sequence of two-element tuples. I'm not even sure why you call urlencode() in the first place. POST method puts the data in HTTP body anyway, requiring no special encoding.

You'll generally get more useful answers if you post a complete code sample instead of snippets without context. I had to assume a lot and construct a test bench for your sample code to get it to do something. Try to reduce the code to the smallest test case possible. Also, in this case you can debug the code on a PC, since no PyS60-specific GUI code is necessary to get the basic things working.

[wirefree101]

Hello,

I am picking up from where the author of this post has left off. The issue is decoding correctly a JPG image send by a phone running a pyS60 script to a server (written in python) running on a PC.

Before addressing the query, I have a general question regarding JSON with Python. Jurgen Schieble's book 'Mobile Python' states the following for understanding JSON:

"On your PC, open the Python interpreter on the same directory where json.py resides."

I have access to json.py and by 'python interpreter' I assume is IDLE (Pyhton GUI/shell). So I have copied the json.py file into C:\Program Files\Python26. However, executing commands such as...

import json
json.write("hello world")


...returns the following error:

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
json.write("hello")
AttributeError: 'module' object has no attribute 'write'

Would greatly appreciate assistance with resolving this minor concern so I may address the larger issue of json decoding phone-server data.

Best regards,
wirefree101

[aaaaapo]

I have a general question regarding JSON with Python. Jurgen Schieble's book 'Mobile Python' states the following for understanding JSON:

"On your PC, open the Python interpreter on the same directory where json.py resides."

I have access to json.py and by 'python interpreter' I assume is IDLE (Pyhton GUI/shell).

That quote probably means you have to open Command prompt and change your "working directory" to the same directory where json.py exists.

So I have copied the json.py file into C:\Program Files\Python26. However, executing commands such as...

import json
json.write("hello world")

...returns the following error:

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
json.write("hello")
AttributeError: 'module' object has no attribute 'write'

If you have now json.py in C:\Program Files\Python26 Do Run...->cmd.exe and issue command
cd C:\Program Files\Python26
and after this command python.exe. Then you can try to import json, python interpreter should now find first the json.py in the same directory.

It is possible IDLE founds some other json, which has no attribute write. You can (in IDLE) try to command help json or dir(json) to see which attributes are available.

[wirefree101]

Greatly appreciate the advise, aaaaapo. The suggested setup has worked well.

As you might recall, the primary concern for this thread is json decoding jpeg. Following is a synopsis:

A photoclient runs on the phone which clicks a photo and sends it to server. The client code is as follows:

import json, socket, camera, appuifw  
  
PHOTO_FILE = u"E:\\Images\\temp.jpg"  
  
def send_photo(name, jpeg):  
    msg = {"jpeg": jpeg, "name": name}  
  
    sock = socket.socket(socket.AF_INET,\  
                         socket.SOCK_STREAM)  
    sock.connect(("192.168.0.2", 9000))  
    out = sock.makefile("w")  
    out.write(json.write(msg))  
    out.close()  
  
while True:  
        name = appuifw.query(u"Photo name", "text")  
        if not name:  
                break  
        print "Taking photo.."  
        img = camera.take_photo(size = (640, 480))  
        img.save(PHOTO_FILE)  
        jpeg = file(PHOTO_FILE).read()  
        print "Sending photo.."  
        send_photo(name, jpeg)  
        print "Photo sent ok"  
print "Bye!"

A server runs waiting for the photo on a PC. Server code:

import SocketServer, json  
  
class Server(SocketServer.TCPServer):  
    allow_reuse_address = True  
  
class Handler(SocketServer.StreamRequestHandler):  
    def handle(self):  
        msg = json.read(self.rfile.read())  
        fname = msg["name"] + ".jpg"  
        f = file(fname, "w")  
        f.write(msg["jpeg"])  
        f.close()  
        print "Received photo %s (%d bytes)" %\  
                (fname, len(msg["jpeg"]))  
  
server = Server(('', 9000), Handler)  
print "WAITING FOR NEW CONNECTIONS.."  
server.serve_forever()

The photo is indeed snapped on the phone (I hear the internals clickin'), and sent correctly and received too by the server. The jpg appears in the same directory as the server script. It is typically 30-40Kb.

However, on opening the image with standard windows picture & fax viewer, a "no preview available" error is generated. Its as if the picture is empty.

Following troubleshooting measures were taken:

- I attempted to open the picture with Photoshop but the error persists
- I attempted running the server using windows cmd line as well as python cmd line with server.py & json.py placed in C:\Program Files\Python26. But the error persists.

Would appreciate advise on addressing the above.

Best regards,
wirefree

[jethro.fn]

The jpg appears in the same directory as the server script. It is typically 30-40Kb.

However, on opening the image with standard windows picture & fax viewer, a "no preview available" error is generated. Its as if the picture is empty.

Sounds like the JPEG might get truncated during transfer. Add debug prints to see the number of bytes of the JPEG data, both on the sending and receiving end. Also see the filesize of your temporary camera image (E:\Images\temp.jpg) and saved file on PC, to determine which part of the chain corrupts the file.

[wirefree101]

Sounds like the JPEG might get truncated during transfer. Add debug prints to see the number of bytes of the JPEG data, both on the sending and receiving end. Also see the filesize of your temporary camera image (E:\Images\temp.jpg) and saved file on PC, to determine which part of the chain corrupts the file.

Appreciate the suggestion, jethro.fn

As suggested, I inserted the following on client side:

print "Sending photo size (%d bytes)" % (len(jpeg))

As for the server, the file size print command is already present. It is:

print "Received photo %s (%d bytes)" %(fname, len(msg["jpeg"]))

The sending & received file size were found to be identical (in bytes).

Interestingly, temp.jpg in E:\images was found and is a valid photo file. I can open it in the phone's image viewer. As for the file sizes, temp.jpg on phone is 28.1KB while the stored file on PC is 28.2KB.

Remain in doubt,
wirefree

[jethro.fn]

As for the file sizes, temp.jpg on phone is 28.1KB while the stored file on PC is 28.2KB.

Ah, you're reading and writing the file in text mode. That corrupts it. Change:

jpeg = file(PHOTO_FILE).read()

to

jpeg = file(PHOTO_FILE, "rb").read()

and

f = file(fname, "w")

to

f = file(fname, "wb")

By the way, it's better style to use open() instead of file().

[wirefree101]

Thx jethro.fn!

Binary was missing.

You ascertained correctly the file corruption earlier itself. Further to that I am interested in your train of thought that established the usage of incorrect file attribute.

Any thought you'd throw on this would be worthwhile.

Best,
wirefree

[jethro.fn]

You ascertained correctly the file corruption earlier itself. Further to that I am interested in your train of thought that established the usage of incorrect file attribute.

Any thought you'd throw on this would be worthwhile.

Well, basically when you said the file was originally 28.1 kB and then 28.2 kB on the PC, that raised a red flag for me. Binary data often contains line feed (code point 10) characters, which expand into carriage return (code point 13), line feed pairs when saving the file as text. Loading binary data as text will also corrupt it but does not change the size nearly as much, as there will originally be fewer carriage return, line feed pairs in purely binary data.

It all comes down to experience. I've been bitten by this text/binary thing in the past and now I know when to look for problems related to it.