How To - Upload an Image to a server (multipart/form-data)

Posted by bpappin, Posted on October 23, 2011, Last updated January 6, 2012

3 votes

Generates multipart form-data suitable for use with Corona SDK's network.request(...) function.

Updated 2011-10-29:
Added some server side example code.
Updated 2011-10-24:
I refreshed the source to make sure I didn't copy it wrong (some people were having trouble getting it to work).
Updated 2011-10-23:
I've replaced the app specific code with a class I wrote to handle generating the POST body and headers.

For the the majority of folks around here who have been asking how to upload an image to their server, nobody had any answers.

The key is an HTTP POST encoded as multipart form-data.
Note: Corona Dev team, this could and should have been in Corona a long time ago!

I've just spent the last several hours working it out, but I've now got something that works.
Keep in mind that this is the code I just hacked together specifically for my own project, and its downright messy (and inefficient), however when I can look at the code again without swearing at it, I'll bundle it into a little module.
In the mean time, maybe some folks that are not as much of a newb to Corona/Lua as myself can help make it more efficient.

Caution:

In Corona SDK you have to set a string as the body, which means that the
entire POST needs to be encoded as a string, including any files you attach.
Needless to say, if the file you are sending is large, it''s going to use
up all your available memory!

Lua Client Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
local MultipartFormData = require("class_MultipartFormData")
 
local multipart = MultipartFormData.new()
multipart:addHeader("Customer-Header", "Custom Header Value")
multipart:addField("myFieldName","myFieldValue")
multipart:addField("banana","yellow")
multipart:addFile("myfile", system.pathForFile( "myfile.jpg", system.DocumentsDirectory ), "image/jpeg", "myfile.jpg")
 
local params = {}
params.body = multipart:getBody() -- Must call getBody() first!
params.headers = multipart:getHeaders() -- Headers not valid until getBody() is called.
 
local function networkListener( event )
        if ( event.isError ) then
                print( "Network error!")
        else
                print ( "RESPONSE: " .. event.response )
        end
end
 
network.request( "http://www.example.com", "POST", networkListener, params)

PHP Server Example

1
2
3
4
5
6
7
8
9
10
11
12
13
$myparam = $_POST['myFieldName'];
echo $myparam;
 
$target_path = "uploads/";
 
$target_path = $target_path . basename( $_FILES['myfile']['name']); 
 
if(move_uploaded_file($_FILES['myfile']['tmp_name'], $target_path)) {
    echo "The file ".  basename( $_FILES['myfile']['name']). 
    " has been uploaded";
} else{
    echo "There was an error uploading the file, please try again!";
}

simon.ong donated this code which has been verified working in a comment below. You will note that he had to manually decode the base64 encoded file because PHP did not do it for him.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$target_path = "/tmp/";
 
$target_path = $target_path . basename( $_FILES['myfile']['name']); 
 
if(move_uploaded_file($_FILES['myfile']['tmp_name'], $target_path)) {
        $src = $target_path;
        $dst = '/tmp/decoded_'.basename( $_FILES['myfile']['name']);
        base64file_decode( $src, $dst );
    echo "The file ".  basename( $_FILES['myfile']['name']). " has been uploaded";
} else{
    echo "There was an error uploading the file, please try again!";
}
 
function base64file_decode( $inputfile, $outputfile ) { 
  /* read data (binary) */ 
  $ifp = fopen( $inputfile, "rb" ); 
  $srcData = fread( $ifp, filesize( $inputfile ) ); 
  fclose( $ifp ); 
  /* encode & write data (binary) */ 
  $ifp = fopen( $outputfile, "wb" ); 
  fwrite( $ifp, base64_decode( $srcData ) ); 
  fclose( $ifp ); 
  /* return output filename */ 
  return( $outputfile ); 
} 

Troubleshooting PHP

  • Review the comments on this page. Most of the hints you need are here.
  • Check your post limit, if the content exceeds the post limit, you won't get the file.
  • With PHP, never post to the same file that the form is in. This is known to cause problems.
  • Check that the "file_upload" option is enabled in php.ini.

Java JAX-RS Server Example

1
2
3
4
5
6
7
8
9
10
11
12
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Produces(MediaType.APPLICATION_JSON)
public Response myFunkyMultipartPost(
                        @FormParam("myFieldName") String myField,
                        @FormParam("myfile") final InputStream fileInput,
                        @FormParam("myfile") final FormDataContentDisposition fcdsFile) {
 
    final BufferedImage image = ImageIO.read(fileInput);
                // do your thing with your image...
 
}

Source:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
-------------------------------------------------
--
-- class_MultipartFormData.lua
-- Author: Brill Pappin, Sixgreen Labs Inc.
--
-- Generates multipart form-data for http POST calls that require it.
--
-- Caution: 
-- In Corona SDK you have to set a string as the body, which means that the 
-- entire POST needs to be encoded as a string, including any files you attach.
-- Needless to say, if the file you are sending is large, it''s going to use 
-- up all your available memory!
--
-- Example:
--[[
local MultipartFormData = require("class_MultipartFormData")
 
local multipart = MultipartFormData.new()
multipart:addHeader("Customer-Header", "Custom Header Value")
multipart:addField("myFieldName","myFieldValue")
multipart:addField("banana","yellow")
multipart:addFile("myfile", system.pathForFile( "myfile.jpg", system.DocumentsDirectory ), "image/jpeg", "myfile.jpg")
 
local params = {}
params.body = multipart:getBody() -- Must call getBody() first!
params.headers = multipart:getHeaders() -- Headers not valid until getBody() is called.
 
local function networkListener( event )
        if ( event.isError ) then
                print( "Network error!")
        else
                print ( "RESPONSE: " .. event.response )
        end
end
 
network.request( "http://www.example.com", "POST", networkListener, params)
 
]]
 
-------------------------------------------------
local crypto = require("crypto")
local ltn12 = require("ltn12")
local mime = require("mime")
 
MultipartFormData = {}
local MultipartFormData_mt = { __index = MultipartFormData }
 
function MultipartFormData.new()  -- The constructor
        local newBoundary = "MPFD-"..crypto.digest( crypto.sha1, "MultipartFormData"..tostring(object)..tostring(os.time())..tostring(os.clock()), false )
        local object = { 
                isClass = true,
                boundary = newBoundary,
                headers = {},
                elements = {},
        }
  
        object.headers["MIME-Version"] = "1.0" 
  
  return setmetatable( object, MultipartFormData_mt )
end
 
function MultipartFormData:getBody()
        local src = {}
        
        -- always need two CRLF's as the beginning
        table.insert(src, ltn12.source.chain(ltn12.source.string("\n\n"), mime.normalize()))
        
        for i = 1, #self.elements do
                local el = self.elements[i]
                if el then
                        if el.intent == "field" then
                                local elData = {
                                        "--"..self.boundary.."\n",
                                        "content-disposition: form-data; name=\"",
                                        el.name,
                                        "\"\n\n",
                                        el.value,
                                        "\n"
                                }
                                
                                local elBody = table.concat(elData)
                                table.insert(src, ltn12.source.chain(ltn12.source.string(elBody), mime.normalize()))
                        elseif el.intent == "file" then
                                local elData = {
                                        "--"..self.boundary.."\n",
                                        "content-disposition: form-data; name=\"",
                                        el.name,
                                        "\"; filename=\"",
                                        el.filename,
                                        "\"\n",
                                        "Content-Type: ",
                                        el.mimetype,
                                        "\n",
                                        "Content-Transfer-Encoding: ",
                                        el.encoding,
                                        "\n\n",
                                }
                                local elHeader = table.concat(elData)
                                
                                local elFile = io.open( el.path, "rb" )
                                assert(elFile)
                                local fileSource = ltn12.source.cat(
                                                        ltn12.source.chain(ltn12.source.string(elHeader), mime.normalize()),
                                                        ltn12.source.chain(
                                                                        ltn12.source.file(elFile), 
                                                                        ltn12.filter.chain(
                                                                                mime.encode(el.encoding), 
                                                                                mime.wrap()
                                                                        )
                                                                ),
                                                        ltn12.source.chain(ltn12.source.string("\n"), mime.normalize())
                                                )
                                
                                table.insert(src, fileSource)
                        end
                end
        end
        
        -- always need to end the body
        table.insert(src, ltn12.source.chain(ltn12.source.string("\n--"..self.boundary.."--\n"), mime.normalize()))
        
        local source = ltn12.source.empty()
        for i = 1, #src do
                source = ltn12.source.cat(source, src[i])
        end
        
        local sink, data = ltn12.sink.table()
        ltn12.pump.all(source,sink)     
        local body = table.concat(data)
        
        -- update the headers we now know how to add based on the multipart data we just generated.
        self.headers["Content-Type"] = "multipart/form-data; boundary="..self.boundary
        self.headers["Content-Length"] = string.len(body) -- must be total length of body
        
        return body
end
 
function MultipartFormData:getHeaders()
        assert(self.headers["Content-Type"])
        assert(self.headers["Content-Length"])
        return self.headers
end
 
function MultipartFormData:addHeader(name, value)
        self.headers[name] = value
end
 
function MultipartFormData:setBoundry(string)
        self.boundary = string
end
 
function MultipartFormData:addField(name, value)
        self:add("field", name, value)
end
 
function MultipartFormData:addFile(name, path, mimeType, remoteFileName)
        -- For Corona, we can really only use base64 as a simple binary 
        -- won't work with their network.request method.
        local element = {intent="file", name=name, path=path, 
                mimetype = mimeType, filename = remoteFileName, encoding = "base64"}
        self:addElement(element)
end
 
function MultipartFormData:add(intent, name, value)
        local element = {intent=intent, name=name, value=value}
        self:addElement(element)
end
 
function MultipartFormData:addElement(element)
        table.insert(self.elements, element)
end
 
function MultipartFormData:toString()
        return "MultipartFormData [elementCount:"..tostring(#self.elements)..", headerCount:"..tostring(#self.headers).."]" 
end
 
return MultipartFormData


Replies

jwwtaker
User offline. Last seen 4 days 22 hours ago. Offline
Joined: 28 Apr 2010

I tried this exactly as you had it written, and the server is not receiving the image file that was attached. the variable isn't even set to be empty its just missing.

emi
User offline. Last seen 3 weeks 1 day ago. Offline
Joined: 18 Mar 2011

Thanks, it looks very good, but after trying it I can't even get the value of "myFieldName" using $_POST['myFieldName'] in PHP.

Am I missing something? I'm not a PHP guru, but using a simple "myFieldName=myFieldValue" as the body it works.

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

I have tested it,
so I know it works on a java back end, but I'm not a PHP guy (much), so I won't know the details of how to solve the problem you guys are having. However I do know that both languages can process multipart/form-data because I've seen it happen.

Don't forget that your processing multipart/form-data not the usual application/x-www-form-urlencoded that Corona usualy sends.
Check that the "file_upload" option is enabled in php.ini.

You can do this also with a simple HTML page, so use some code like this to test the PHP side of things. Once you *know* the PHP side is working, try it in Corona:

1
2
3
4
5
6
7
8
9
10
11
<html>
<body>
<form action="http://localhost:8080/yourpath"
                  method="POST" enctype="multipart/form-data">
                                <input type="text" name="alias" />
                <input type="file" name="avatar" />
                <input type="submit" name="submit" value="upload" />
            </form>
 
</body>
< /html>

Here are some links and threads that should help:
http://www.tizag.com/phpT/fileupload.php
http://stackoverflow.com/questions/1075513/php-parsing-multipart-form-data

FYI: if you happen to be using Jersey in Java for your REST services, as I am, the code would look something like:

1
2
3
4
5
6
7
8
9
10
11
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Produces(MediaType.APPLICATION_JSON)
public Response myFunkyMultipartPost(
                        @FormParam("alias") String alias,
                        @FormParam("avatar") final InputStream picInput,
                        @FormParam("avatar") final FormDataContentDisposition fcdsFile) {
 
                // do your thing with your data...
 
}

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

use the $_FILES var not the $_POST var to get the files.

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

Anyone else using PHP that can test this?
I'm concerned that the two folks that have tried so far can't get it to work.

simon.ong
User offline. Last seen 2 hours 22 min ago. Offline
Joined: 30 Apr 2011

Hi bpappin,

I can't get it to work too. It uploads successfully but the file that is uploaded is corrupted.

I did a comparison of hex dumps with TextWrangler of the corrupt file with another one that worked (using http://developer.anscamobile.com/code/upload-binary-corona-php-script) and noticed the following:

- The working file was smaller than the corrupt file.

- A portion of the corrupt file from the beginning matched the middle of the working file till the end.

- There seems to be some header missing in the corrupt file (that includes ASCII "JFIF") that is present in the working file

I have no idea why, but this is just what I can see.

Anyone else got this to work?

Thanks,
Simon

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

That is odd.

The file is base64 encoded in order to be sent through the corona api.
I'm wondering what php does on the server side to decode it.

I know the multipart/form-data is encoded properly on the corona side, because I've tested it repeatedly in the process of development, the only difference would be the server side decode.

a clue is that the file you get is larger. that may mean that php did not decode the content for you.

Some things to check:
- make a plain HTML form and try the upload. Does it work then?
- make sure you max file size can handle not just the file size, but the entire body (including all the params and their headers).
- You can print multipart:getBody() before you set it on params.body so you can see what the code is sending. might help you debug.
- I don't know if PHP will automatically decode the base64 data (I think it does). That might have to be a manual step. but i don't know that.
-In the lua code, try removing the mime.wrap() param where the file is encoded. maybe PHP doesn't like the wrapped base64 data. That param wraps the data at 80 chars for readability but should not really be required for this to work.

I just did a google search for "php multipart/form-data file corrupt" and found a whole bunch of stuff on it, include some official bugs that were fixed at some point, so I'm now wondering if you are running into one of those. Some of them specifically mention the file on the server being larger than the sent file.
Essentially they boil down to the multipart/form-data handling to be buggy in PHP for some versions, particularly in Linux.

I do plan to set up some PHP tests of this to test it myself, but I haven't had time recently to get back to it :)

simon.ong
User offline. Last seen 2 hours 22 min ago. Offline
Joined: 30 Apr 2011

Thanks for your reply, I am using a Linux server but PHP is almost the latest version, not sure if the bug still exists. Will check out your suggestions above and revert if I manage to get it working. Thanks again ;)

Dhennrich
User offline. Last seen 3 weeks 5 days ago. Offline
Joined: 20 Jan 2011

Can I decrypt the file before saving him? in php

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

I'm not sure what you mean by "decrypt" the file?

This code doesn't encrypt it (although you could likely add that feature).
It does base64 encode it though, so it can be sent as plain text in Corona.

I don't know if the file coming out the other side will be automatically decoded by PHP or not. You *may* need to do it manually.
You should be able to tell by looking at the file content if it's still base64 encoded when you get it from disk.

Dhennrich
User offline. Last seen 3 weeks 5 days ago. Offline
Joined: 20 Jan 2011

I can upload the image fine and it shows in my web site but I can't open to see what is inside, so I can't see my picture

can you help me?

Thanks

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

Hmm... take a look at it on the file system.
is it still base64 encoded?

If so, try decoding it.
You might have solved half the trouble people are having in PHP if that is the only issue your having :)

simon.ong
User offline. Last seen 2 hours 22 min ago. Offline
Joined: 30 Apr 2011

bpappin / Dhennrich,

Thanks for the clue, I had problems with this before too, the following works for me, tested with a PNG (image/png) file. Tested with a text/plain file too.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<?php
$target_path = "/tmp/";
 
$target_path = $target_path . basename( $_FILES['myfile']['name']); 
 
if(move_uploaded_file($_FILES['myfile']['tmp_name'], $target_path)) {
        $src = $target_path;
        $dst = '/tmp/decoded_'.basename( $_FILES['myfile']['name']);
        base64file_decode( $src, $dst );
    echo "The file ".  basename( $_FILES['myfile']['name']). " has been uploaded";
} else{
    echo "There was an error uploading the file, please try again!";
}
 
function base64file_decode( $inputfile, $outputfile ) { 
  /* read data (binary) */ 
  $ifp = fopen( $inputfile, "rb" ); 
  $srcData = fread( $ifp, filesize( $inputfile ) ); 
  fclose( $ifp ); 
  /* encode & write data (binary) */ 
  $ifp = fopen( $outputfile, "wb" ); 
  fwrite( $ifp, base64_decode( $srcData ) ); 
  fclose( $ifp ); 
  /* return output filename */ 
  return( $outputfile ); 
} 
?>

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

@simon.ong

Nice!
If you don't mind, I'd like to include your solution as the example above.

simon.ong
User offline. Last seen 2 hours 22 min ago. Offline
Joined: 30 Apr 2011

@bpappin, sure go ahead. Thank you very much for your class too! :)

alabelson
User offline. Last seen 5 weeks 22 hours ago. Offline
Joined: 14 May 2011

For people having an issue with this, try changing the following in the php file on the server side:

$dst = '/tmp/decoded_'.basename( $_FILES['myfile']['name']);

to

$dst = '/uploads'.basename( $_FILES['myfile']['name']);

The original script would put the base 64 decoded file in the /tmp directory and not in the /uploads folder, where it is expected.

mpappas
User offline. Last seen 20 hours 33 sec ago. Offline
Joined: 31 Jul 2011

I'm having an issue with this on android.

When I post to the server, my listener gets called and it has a response of "Java.Lang.Double"... (I don't believe the http bits are making it out of the phone...)

It fails for small size and large size images. However, everything works perfect on the OSX simulator and the iPhone.

Anyone got this to work on the android? Did you have to modify anything?

BuxPod's picture
BuxPod
User offline. Last seen 57 min 43 sec ago. Offline
Joined: 14 Nov 2011

+1 - Not working on android

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

Sorry, I abandoned Corona a while ago because it kept getting in the way and things never seemed to work, while hiding the actual code from you... so I can't help you debug.

If you do find out what is wrong, post it here so others can benefit from your experience.
It likely has to do with the http client on android, i seem to remember that there needs to be some support libs included.

BuxPod's picture
BuxPod
User offline. Last seen 57 min 43 sec ago. Offline
Joined: 14 Nov 2011

hi bpappin,
all night long and this monrning just to perform basic authentication on an server RestFul service.

Still get network error on android. tryed everything.
Simply, it doesn't work.

We are going to loose this job after 48 hours trying to understand if it can be accomplished with corona, stucked on this network error.

The log in with native ios and with restful consuming clients works perfectly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
local json = require "json" 
local mime = require "mime"
 
 
        local params = {
                headers = {
                        ["Content-Type"] = "application/x-www-form-urlencoded",
                },
                body = "identifier=username1&password=password1"
        }
        
        local function networkListener(event)
                if ( event.isError ) then
                        print( "ERRORE NETWORK")
            else
                        print (event.response)
            end
        end
 
        network.request( "http://username1:password2@api.example.com/login", "POST", networkListener,params)

BuxPod's picture
BuxPod
User offline. Last seen 57 min 43 sec ago. Offline
Joined: 14 Nov 2011

The problem seems to be with the /something after the url in the request.

(tryed on corona 786, and corona 812)

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

As I mentioned before, android may need additional libraries in order to handle multipart post.

For example, see:
http://stackoverflow.com/questions/2017414/post-multipart-request-with-android-sdk

I have no idea if Corona even allows you to add libraries.

This is as far as I can help you with the problem, regardless of how urgent it is for you.

bpappin's picture
bpappin
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 19 Jan 2011

One other thing, you code shows you using "application/x-www-form-urlencoded" when this class is all about "multipart/form-data".

Are you maybe just confused one what you are doing or posting to the wrong place maybe?