Thank you!

[hqkirkland]

I don't really know how else to post this, so I'm sorry if it bugs y'all. But the one thing that kept me reverse engineering a server for my favorite online game, "Spineworld", were the thanks and appreciation from fellow players who longed for its return. And I want this project to be appreciated in the same way.

The first game I ever played was Junkbot, around age 4, no doubt. Then there was Spybot, then "Spineworld", and Habbo. I was really worried that these games and their technical secrets would be lost as their plugin faded into obscurity. I can't tell you how excited I am to think that people are working hard on a project that could open them up a little.

I can't do much in the way of decompilation, nor am I very good at low-level things like this, but I have some good knowledge of the way Spineworld's client operated, as well as a few versions of it on hand. I feel like it's a good project to sample a lot of functionality from (i.e. external cast/data/sound loading, networking, xtra usage). If it helps I'm happy to provide any resources and intel on it, should you choose to use it to explore certain aspects of Shockwave.

[Brian151]

Oh, I quite appreciate this. but, I understand your reluctance, I am much the same way.

Spybotics is under the umbrella of what I unofficially-internally refer to as "Project Resurrection"
This is a group of my programming/hacking projects all aimed at bringing old games back with the newer tech, or extracting/archiving everything I possibly can, at the very least.
https://github.com/Brian151/Various-Projects/tree/master/SpybotNightfall
OpenShockwave, at least from my side, is primarily motivated by the efforts to preserve said game.
One tidbit: The release of those materials exists under not only legal gray area, but authorization. I neither am allowed nor disallowed to share. While able to contact some of the former employees, I haven't gotten anything major (aside from the CCTs) because they either don't have it, or can't/won't share it. Which, it totally understand

I am still pretty new to low-level stuff, myself. Any help is welcome, Shockwave is quite possibly more broad than Flash, by far. Flash, most specialized code is compiled into a given SWF. Shockwave has Xtras, and also is very supportive of 3D graphics. It also is very familiar with the embedding of arbitrary binary files, something Flash only picked-up kinda recently. (AS3) However, again, Flash doesn't rely on a plugin system, you either have module SWFs, or the main SWF to parse custom formats. So, reversing some arbitrary format (or encryption) is fairly trivial.

[Qiangong2]

I'm so happy you are working on this! Than you! I grew up playing Junkbot, Spybotics, and Backlot and I hope this project makes it possible to enjoy these games fully again :)

[Brian151]

I tried so hard with backlot...
That game was just confusing...
I also tried playing it according to a walkthrough, and it crashed at some point
Still want to give it a shot sometime

Junkbot I maybe could get around 25% of each game
Along with World Builder 2, it was my first of LEGO's Shockwave Games

I discovered Spybotics shortly before LEGO pulled the plug. I got to the end, albeit through serious trial and error. It got pretty hardcore, though, so I still never finished it.

I will be honest, I have plans to rip from all of the LEGO Shockwave games, particularly those by Gamelab.
However, I will focus my efforts primarily on Spybotics and WorldBuilder for porting. Backlot is beyond my current capabilities, and I'm not sure about Junkbot. I might also tackle Inventor Saves the Day. The thing is, I have a pretty hefty list of games I want to port as-is... even the simpler ones take time. Honestly, I should give spybot some more love, I have enough material/knowledge to start actually making something...

[MrBrax]

I just had to google Spybotics, as i vaguely remember that name.. and wow, i had one of those remote controlled things as a kid! Although, i never figured out how to play that game, and got so disappointed it was just some simulated thing

[Brian151]

Spybotics had two games

there was nightfall incident, and something about hacking frequencies...
the latter always confused me

the fact they're sims doesn't bother me, i like sims. what surprises me, though, is how nightfall really fits-into the theme besides hacking, and a few shots of the product. other than that, i'm pretty sure the entire game was concocted by gamelab. there's barely anything that truly ties it to lego, or spybotics... not that i'm complaining, it' just weird...

[hqkirkland]

That's so funny you mention that frequency game, because I went on a searching adventure for a set of archives containing the Lego games (seems I didn't immediately download it when I first found it. Hate it when that happens!), in the hope of finding it.

http://nodebay.com/archives/Lego/Wi-FiCracker/DSWMEDIA/

[hqkirkland]

I always had that same impression from Spybotics, that it was a stand-alone production rather than a spin-off from a line of products.

What Lego probably wanted to do was test how influential the game would be towards generating sales, rather than a product line generating game plays.

[hqkirkland]

Also I didn't mean to close this.

[Brian151]

right, wi-fi cracker was the name
honestly, there exists a much larger archive of lego games, but i'd rather not give it too much attention
(especially with the hostile environment towards such things, now)

your theory makes sense

and no worries!

[MrBrax]

(especially with the hostile environment towards such things, now)

did i miss out on anything?

[Brian151]

Not really?
I just... there's all these lawsuits, takedowns, C&D's going-around... (and attempts to further write + enforce draconian policy in regards to copyright/trademarks/patents, and especially in secret)
IDK how public this other site is, but their complete archives are in the GB file sizes.
Not only that, but they're quite expansive.
It'd be a huge loss if that were targeted, I'd rather not even think of it.

[jdjakub]

Huge props to you for doing this. (I'm mainly invested in the Spybotics game, but all the others were great too)

I want to suggest something, sorry if it's naive or if you're already doing it but I just found out about this project :)

A while ago I was messing around with Spybotics in Cheat Engine. I found that the Lingo scripts were plainly visible in memory, at runtime, whereas they must be compressed or obfuscated in some way in the file on disk.

It sounds like you've already succeeded in getting all that and other resources from the files, but still: is there any tracing / step-debugging of the runtime execution going on? If that were feasible, we could learn a lot about how the data structures are actually used and what the bytecode does. It would be very low-level debugging, though, so it might be too painful to be useful. Just a thought anyway.

[Brian151]

thanks!
(indeed it's my primary focus. Would love to collaborate with someone, tho... )

clarify 'plainly visible'?
on the final file, they're compressed via ZLIB, and the whole file is also structured just a tad differently. it's rather annoying, actually...

I haven't quite gotten everything, I'm still missing a ton from "visual" cast, and I haven't gotten all the script names, yet... @tomysshadow has done some low-level debugging, but I'm not sure if with any REAL-WORLD examples... Most of this project is very painful, indeed. Knowing the data structures would be neat, but I somewhat debate its usefulness. I'd certainly like to know how scripting works, though, or at least, to have such knowledge available...

[jdjakub]

clarify 'plainly visible'?

What I mean is, I could see actual (what I presume is) Lingo text source code in some parts of memory. I was surprised at first, because nightfall.dcr (or whatever it is) doesn't contain a trace of such human-readable text, and I didn't know whether that was due to compression, or encryption, something else, or a combination of the above. But it makes sense, because the program needs the un-garbled version to execute, not the garbled version.

Although... I didn't know about the fact that it uses bytecode. But some source code is there in memory at runtime. It was describing the structure of the netmap. Snippet:

[
  [#ID: 1, #company: 0, #type:"SMART", #icon:1, #name:"SMART HQ", #path:[753,792,775], #state:#WIN, #data:[#security:1, #reward:400, #creditItems:0] ],
  [#ID: 2, #company: 7, #type:"WAREZ", #icon:1, #name:"LEO'S SHOP", #path:[753], #state:#ON, #data:[#security:1, #reward:400, #creditItems:0] ],
  [#ID: 3, #company: 3, #type:"BATTLE", #icon:1, #name:"TECH SUPPORT", #path:[775,782], #state:#ON, #data:[#security:1, #reward:400, #creditItems:0] ],
  [#ID: 4, #company: 3, #type:"BATTLE", #icon:1, #name:"Club Center", #path:[782,755,768], #state:#OFF, #data:[#security:2, #reward:600, #creditItems:0] ],
...
]

So maybe 'structural' code is kept as text, and the 'real work' is done via bytecode? Or maybe all the code is there and it compiles to bytecode on startup? I'm just speculating here. As usual, it's a mystery :)

I've done some ASM-level debugging before in OllyDbg. But that was with assembly programs that I'd written, and whenever I dared to debug the guts of some everyday application written in C++ or something, I would give up after about ten seconds. Still, I'm tempted to give it a go in this case...

I'd certainly like to know how scripting works, though, or at least, to have such knowledge available...

So, when you say 'scripting' ... I assume you mean the Lingo scripting in Shockwave specifically? I'm just as in the dark as you on any of the details of Shockwave :/ but figuring it out is the "fun" part!!

Where should I start to find out what stage this is all at? I looked in the docs folder but I got a bit lost after that. What tools have you used / are you using? (I have HxD, but I'm not familiar with anything to do with Flash yet :S)

Also, I saw on this thread that you said you are having trouble with stacks or something. Do you mean: Lingo bytecode is stack-oriented, i.e. push, push, add, push, push, sub, mul? And you want to decompile that into some HLL like JavaScript?

[hqkirkland]

That's not really code at all. It's simply a structure of text that's stored in memory. It's parsed by the game's code; it's basically a data format, most likely unique to Spybot.

They even have a dedicated list-parsing tool due to this find. But the fact is, this is not Lingo at all, and just something that you can find in memory.

[jdjakub]

It just looks like property list structures in Lingo syntax, where the keys are symbols i.e. beginning with #

[hqkirkland]

Yeah but still, Lingo clearly stores those as plaintext, instead of its own data structure, I guess. But yeah, it's been found.

[Brian151]

lingo bytecode doesn't store them as plaintext
they'd be stored as something like:

NEWLIST
PUSH #symbol
PUSH value
;repeat however many times
NEWPROPLIST listSize

roughly, i vastly over-simplified the process...

what you're referring to are text cast members found in places like leveldata and textdata