Just when I think this place is not going to one up itself we get something like this...
I have never heard of this device/setup before this thread and it is not mentioned in the hardware docs I have seen.
I would look at the sewing machine itself and see what it has as far as chips -- it is probably old enough that a dedicated chip would do something and that would save a whole load of headaches if you can get something on that instead.
Sewing machines today are odd things and fairly manufacturer specific (I might have upset the sewing machine shop looking to make things from their machines) and most computer controlled machine tools will use something called gcode which has been around since the 50s, though some flavour of computerised automation goes back further still.
I can't rule out something more basic either baked into the machine or a simple custom serial lead fed command either.
As far as vectors on a GBC then I would be shocked if it was. Prebaked, maybe with a scaling command, I would bet heavily on. On the other hand https://kotaku.com/5800606/this-might-be-the-weirdest-game-boy-game-ever-made
mentions some form of editing, though if the video in it is anything to go by it is might just be letters which is still prebaked.
I am also seeing videos on its usehttps://www.youtube.com/watch?v=Bd742Tp7b2U
(about 13 minutes it starts)https://www.youtube.com/watch?v=K-Epj2_eJdA
However I am going to have to return to those when it is not 1am. Though I did skim through very quickly to see some of the custom stuff and it still looks like a prebaked selection. With that said then if such features are available at some level you probably can make a custom image come out, though the techniques will resemble level editing more than they resemble tile editing.