News: 11 March 2016 - Forum Rules

Author Topic: What type of codes you use to deal with tables?  (Read 4350 times)

jjjewel

  • Jr. Member
  • **
  • Posts: 26
    • View Profile
What type of codes you use to deal with tables?
« on: June 14, 2012, 12:48:48 pm »
I'm just wondering what kind of codes people use to deal with encoding tables.

Ex. If you have a table like this
0000=A
0001=B
0002=C
0003=D
 and so on.
(This is just an easy example. In most cases, I'm dealing with Japanese games with full Shift-JIS table or sometimes dual tile encoding tables.)

What I've tried is splitting the text file for table into lines. Then split each line into code and text using String Split() function and add them to hash table. (I also tried string array with codes() and text() and loop through them to find a match but I read from somewhere that it's slower than hash table.)

Anyway, this method is still kind of slow. And I have used other programs that also deal with custom tables and they usually process in less than a second when my program takes like 10-20 seconds.

If anyone can give me some hints on how to improve it, that'll be highly appreciated. (I'm using Visual Basic Express and I don't know much about advanced programming codes, so please explain in details if it needs some complicated coding. Thank you.)

LostTemplar

  • Hero Member
  • *****
  • Posts: 910
    • View Profile
    • au-ro-ra.net
Re: What type of codes you use to deal with tables?
« Reply #1 on: June 14, 2012, 01:37:03 pm »
What exactly is slow? Loading the table or using the table (e.g. finding the corresponding code to a character)?

jjjewel

  • Jr. Member
  • **
  • Posts: 26
    • View Profile
Re: What type of codes you use to deal with tables?
« Reply #2 on: June 14, 2012, 03:17:35 pm »
What exactly is slow? Loading the table or using the table (e.g. finding the corresponding code to a character)?

Both, maybe? I didn't time it or anything yet, but the overall process is quite slow.

First of all, I think I should improve the loading first. In most hex editors, when I clicked apply table, they usually process it in an instant. While for my program, it takes some time to split the lines, then split the code/text pair then match the code and output to text, etc. So I think I must've done it in an ineffective way.

Here's the code I used for loading my table to hash table.

Code: [Select]
    Private Sub MakeTable()
        If m_table Is Nothing Then
            Dim myLines() As String = File.ReadAllLines(tableFile)
            Dim myTable As New Hashtable
            For i As Integer = 0 To UBound(myLines)
                If myLines(i) <> "" Then
                    Dim temp() As String = Split(myLines(i), "=", 2)
                    myTable.Add(temp(0), temp(1))
                End If
            Next
            myLines = Nothing
            m_table = myTable
        End If
    End Sub

(I check if the table is nothing so if it has already loaded once, I don't need to re-load it. It'll save time to create hash table. But still, it's not that fast. P.S. I use Stringbuilder to output my text)
« Last Edit: June 14, 2012, 03:29:22 pm by jjjewel »

Karatorian

  • Sr. Member
  • ****
  • Posts: 381
  • "Gotta get get ... 6502"
    • View Profile
    • Studio Karatorian
Re: What type of codes you use to deal with tables?
« Reply #3 on: August 10, 2012, 09:09:42 pm »
Well I don't know that much about VB (I assume that's VB, if not, it just shows how little I know), but from what I can tell, I don't see any glaring errors in your algorithm. You could try reading the lines one at a time inside the loop, which would use less memory, but I don't know if that would speed things up or not. (and it might actually slow things down...) However, if performance is unacceptable, the fault is probably with some of the built-ins you're calling. Perhaps someone who knows the language better could suggest some more performant alternatives.

However, the standard table file format was designed with small character sets in mind. Shift-JIS is huge and you might be better off using a dedicated character encoding library if you have to handle it a lot. If you can find one to which it's easy to add your own encodings, retaining traditional table file support should be easy as well.
Current ProjectsFinal Fantasy EngineSMB Special for NESStudio Karatorian
@loop: lda (src),y — sta (dst),y — iny — bne @loop — inc src+1 — inc dst+1 — dex — bne @loop

Rhys

  • Hero Member
  • *****
  • Posts: 710
    • View Profile
    • CN
Re: What type of codes you use to deal with tables?
« Reply #4 on: August 11, 2012, 02:06:18 am »
Looks like you're doing it fine to me, one thing you might want to try is reading the table file line by line in your loop, but that'll more be for saving memory than saving processing time

jjjewel

  • Jr. Member
  • **
  • Posts: 26
    • View Profile
Re: What type of codes you use to deal with tables?
« Reply #5 on: August 11, 2012, 09:23:24 pm »
Thank you for your replies. I ended up using my old programs. (Kind of get used to its speed after using it for a while.) Anyway, I'll try to see if I can improve the speed and memory usage for it. Thank you. :)