Optimization for loading data into tables... caching calculation results and more

No replies
OderWat
User offline. Last seen 28 weeks 1 day ago. Offline
Joined: 4 Jun 2010

I ran into this in my project and found some posts in the forum which where asking for optimization of loading data into tables.

What I want to show here is pretty basic or at least should be for every long term programmer but it may be inspiring for people which have less experience (and esp. not from the old times where you had to code the code.. lol)

I explain the technique I use with an example from my project:

There is a calculation which needs to be done multiple times for every "turn" in my game which is pretty expensive but has one parameter and limited values to them. The calculation is pretty expensive because it needs bitwise operations which are not supported in corona and have to be simulated.

Think of this as function "erg(x)" which returns a table (a list of results)

To speed this up one can use "x" as a value into a table which is precalculated and holds all possible results from the calculation.

So my first optimization was to introduce a table which holds all the results with "x" being the hash for this..

This table is generated at the start of the application and needs to be calculated only once. Cool! This speeds up the application but has the drawback of a long delay at program start.

To speed this up you may come to the idea to write this table into a data file and load it into a table when the application is started. This is of course a possibility but it can be done much better!

The table you wanna create from loading your data is basically a lua table :) which can be expressed by lua code. And it is "static" and not changing!

Having this in mind the solution is easy: Just create lua code which creates the table and require this code into your project!

I show you some code as example because it may be tedious to get the code generation work in a nice way with formatting and with the right syntax.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
function Foo:genErgCacheLua()
        local fh=io.open("ergcache.lua","w")
        -- we use "module(...)" no need for the package.seeall!
        fh:write("module(...)\n\n")
        fh:write("ergcache = {\n")
        local first=true
        local k,v,n
        for k,v in pairs(self.ergscache) do
                if not first then
                        fh:write(",\n")
                else
                        first=false
                end
                fh:write("\t['"..k.."'] = { ")
                for n=1, #v do
                        fh:write(v[n])
                        if n ~= #v then
                                fh:write(", ")
                        end
                end
                fh:write(" }")
        end
        fh:write("\t}\n")
        io.close(fh)
end
 
function Foo:loadErgCache()
        -- following looks strange but works :)
        self.ergcache = require "ergcache".ergcache
end

Of course there are assumptions I make for my very own "ergcache", name and classes and so on. But the concept should be clear now! Don't forget to copy the file to the resources of your project :)

I hope this will help some of you to get more speed and inspires you to use more complex techniques in your programs to speed up things which could not be done otherwise!

P.S.: This is also posted in my Blog (http://bit.ly/bw5k9W)