Save states were available in hardware for a few machines, I think I pulled one such device apart and stuck pictures up here once. Similarly cheats were available in software/initial loaders/breaks in things like the commodore 64 (see poke and peek). For cheats explicitly in emulators then debugger support has been available for a while and the ability to reach out and touch the emulated system's virtual memory would have been part of that, I don't know when a more "formal" approach might have been adopted.
Rewind is the logical conclusion of save states to me, I vaguely recall seeing a macro once that would hit savestate three times and give you a choice and another that would automatically savestate every so often. I don't know if it is was a lack of inspiration or a lack of system power that held things back though.
Plugins... is that a thing worth noting? From a programming perspective it feels no different to using a DLL file or maybe using someone else's sound core or something like that (though if that is statically compiled into the program then I will note a minor difference). Alternatively it is arguably the continuation of the game specific bodges and hacks that we are still dealing with the fallout from today in some cases.
Fast forward and overclocking for most consoles are the same thing. I am not sure when it would have hit emulation as a thing but at the same time most old consoles are synced with their clock (why main clocks often vary between NTSC and PAL regions). Emulating is inherently going to want to match speeds for the output machine. Again I don't know if lack of grunt or lack of inspiration would have held it back, especially if frameskip was still a necessary thing. Disable vsync and skip vblank and whatever else would have effectively done it, even if someone would take a while to later bind it to spacebar or something, and I recall seeing options for that way back when.
Netplay on home consoles is basically video and control streaming, for handhelds it might be something more as you have some custom communications and for modern stuff it might well just being able to emulate enough of the system to allow the servers to think it is real.
I don't know if anything predated Kaillera at this point, or if someone used an earlier version of RDP or something to make it happen but I would not be surprised as a lot of people had fun with business machines at the time and that was another point in time when remote computing/mainframe thinking/network terminal was on the rise.
For the handhelds then http://problemkaputt.de/gmbhist.htm
"v1.1 10/97 much more carts working, 2 player link, better sound, ems used"
For my money the main development I would really want to note would be the introduction of dynamic recompilation, or dynarec in some circles.