Assuming there's not already an English font in the game, then yes, you'll have to replace the Japanese font graphics with an English one.
In really simple terms, the game reads in the numbers given to it, and draws the tile that's assigned to that number. Tile 65 can look like the letter 'A' or the hiragana 'あ' - the game doesn't care, it just draws that tile whenever it sees the number 65. The number itself has no inherent meaning.