SA-MP Forums

Go Back   SA-MP Forums > Other > Everything and Nothing

Reply
 
Thread Tools Display Modes
Old 08/01/2018, 01:30 PM   #1
Misiur
High-roller
 
Misiur's Avatar
 
Join Date: Jul 2009
Location: Poland
Posts: 2,522
Reputation: 550
Default Loaded AMX viewer

Hi!

Some libraries are using black magic and PAWN quirks to generate code at runtime (see amx_assembly). While this is wonderful and allows for expanding PAWN power beyond what it was supposed to be able to do, when something goes wrong, crashdetect returns complete garbage and debugging stuff is neigh impossible. Sometimes crashes are platform specific, code works fine on windows, but fails on linux boxes. So, what I need is a dumper of representation of current state of AMX loaded into samp server memory.

Did anyone already write such a program?

If not, I'm asking you for guidance in starting such an endeavour. My goal would be to write a GUI program running alongside running samp-server, with live AMX viewer. Maybe in future also current PAWN virtual machine state as well (current instruction and stuff), maybe even live memory editing (already possible in-game with samp-introspect mind you). I'd like to write it in C++, becuase why not learn something new along the way (I can mostly understand what is written in C++ (not necessarily what it does though), as long as there are no magic compiler-extension intricacies).

What I think I will need:
1. Some kind of DLL hooking (is there a linux .so equivalent?), or maybe plugin loaded the "official" way which will communicate with the debugger - but are there limitations for what plugin can read? More experienced PAWN coders than me used bugs in AMX VM to read (and write) whatever they want, however I don't know anything about writing plugins more than from what I have read in plugins section starting guide.
2. Cross-platform GUI. I think I'll need something like GTK (#edit: I meant Qt)? I don't know, I have superficial knowledge of those libraries from more than 5 years ago, maybe now there's some better contender.
3. I do not need a C++ tutorial, there's plenty of them on the internet.
4. Huh, I thought there would be more, but for MVP that's enough.

I know how to use OllyDBG, albeit slowly and not always with success.

I know how to write a dumper in PAWN, just read whole memory using #emit when OnRuntimeError is called, but I partially want to learn something new, and, well, adding debugging code directly in code being debugged adds noise (and when debugging someone else's problem, I don't want to edit the AMX provided).

I don't care about 3rd party plugins internal states, as there are better debugging tools for that.

Of course code will be on github under MIT license.

Quite a wall of text. So, what should I read first to start this project? What limitations are there in my way? Which crossplatform GUI library would be best for this specific use-case? Is using official plugin to do this is the best way? Which way would be best to broadcast read data to the debugger? Sockets for remote debugging?

Thank you.
Misiur is online now   Reply With Quote
Old 08/01/2018, 01:47 PM   #2
DRIFT_HUNTER
High-roller
 
Join Date: Oct 2009
Posts: 2,142
Reputation: 157
Default Re: Loaded AMX viewer

I guess plugin itself would be able to access any part of samp's server memory (including other plugins) because it would be loaded into process itself and since you will use c++ you can always unprotect memory space for access (mprotect function on unix systems, virtualprotect function on windows).

Launching gui alongside could be a bit of work since you want cross platform. By the way GTK is perfectly fine, but for some reason most people prefer QT (guess its html5 and other quirks). Just use what you are comfortable with, nobody cares about how it looks as long as its functional.
__________________
Путин here,
Путин there,
Путин просто everywhere.


Any PM's that include question about any kind of help will be ignored.
Use appropriate boards for that
DRIFT_HUNTER is offline   Reply With Quote
Old 08/01/2018, 02:11 PM   #3
Y_Less
Spam Machine
 
Y_Less's Avatar
 
Join Date: Jun 2008
Location: 629 - git.io/Y
Posts: 14,542
Reputation: 2974
Default Re: Loaded AMX viewer

This is quite freaky timing. I've been thinking of something similar for a while, but hadn't actually mentioned it to anyone until yesterday when I was talking on Discord with Amagida. What I was thinking is not the same thing, but would achieve what you want in a slightly different manner almost as a side-effect:

1) A plugin, similar to crashdetect or JIT, in that it entirely takes over AMX code execution. IMHO this is much simpler than trying to hook in to the existing code execution in the server, and the AMX VM is open-source.

2) Again, similar to crash detect, is adding code to load the AMX debug information if any exists, to map file line numbers to assembly instructions.

3) Augment the main VM outer loop with a little thing to output the current instruction number at every step, plus any modified registers or memory locations. That also shouldn't be too hard - simply change the "_W" macro to not just write the memory but also note the change, then at the end of the loop look at which registers changed and output those (augmenting every instruction to do this would take a lot of repetative effort, but there are only a few registers so checking them all at the end is not hard).

4) An external tool similar to the one you already said, showing the currently executing instructions, plus memory pulled from the plugin.

5) This together enables one feature we have always SORELY lacked - SINGLE STEPPING!

Because the currently executing instruction is now controlled by a plugin, when to execute it is entirely in our hands, or the hands of the external GUI:

Step one instruction - easy.
Step one line - not hard, break at the next "BREAK" or "PROC" OpCode, and if the GUI loads source code using debug information to match that to assembly we can follow code through.
Step over call - similar to the last one, but setting a breakpoint at the next "BREAK" in the assembly, rather than just stopping at the next "BREAK" executed (since with calls they might not be the same thing).
Breakpoints - Also easy.

"Step over" and "breakpoint" are actually the same thing. Breakpoints would be stored in the plugin, and an event raised when their addresses are hit, with "step over" simply being the GUI setting a hidden breakpoint on the next "BREAK". The other steps would be more plugin controlled - either raising an event every instruction or every special instruction.

So in short - write a plugin to do your thing, and I'll be VERY interested in assisting as a means to get to my idea.
Y_Less is offline   Reply With Quote
Old 08/01/2018, 05:38 PM   #4
[HLF]Southclaw
High-roller
 
[HLF]Southclaw's Avatar
 
Join Date: Apr 2009
Location: England
Posts: 4,769
Reputation: 1293
Default Re: Loaded AMX viewer

This sounds like an interesting project!

During developing the vscode-pawn extension for Visual Studio Code (my new favourite editor) I did quite a bit of research into its extensibility and the API is impressive! I think it would be brilliant to write this application as a standalone console application so you don't need to worry about the GUI (I find GUI is often overlooked as simple, it usually turns into a component just as big as the core application!). The Golang debugger was actually built just using this API along with other languages like TypeScript, JavaScript and C#.

That also means you don't need to worry about cross-platform UI, it also means you could run the debugger process on a server and control it via the debugging protocol from your personal machine.

Here is some information on the debugging protocol: https://code.visualstudio.com/docs/e.../api-debugging

I'd be willing to help with the vscode side of things if you decide to go this route. I've already got an extension released for vscode and it would be really cool to add debugging support into that.

One last thing too, writing it as a standalone application also means you (or others) can integrate into other editors such as Atom - though I'm not sure what their debugging API (if there is one) looks like.
__________________



[HLF]Southclaw is offline   Reply With Quote
Old 08/01/2018, 07:49 PM   #5
maddinat0r
Gangsta
 
maddinat0r's Avatar
 
Join Date: Jun 2010
Location: Germany
Posts: 888
Reputation: 460
Default Re: Loaded AMX viewer

There is actually a mechanism already implemented into the abstract machine, called the "debug hook". It's basically a callback function that gets called on every BREAK opcode. Check the PAWN implementers guide on this for more information. There is also a whole chapter dedicated to debugging support.
maddinat0r is offline   Reply With Quote
Old 08/01/2018, 11:36 PM   #6
Misiur
High-roller
 
Misiur's Avatar
 
Join Date: Jul 2009
Location: Poland
Posts: 2,522
Reputation: 550
Default Re: Loaded AMX viewer

Alright guys, small step for me, giant step for... well, nobody yet.

Armed with newly found plugin development tutorial and implementer's guide example code (with AMX_ERR_EXIT substituted with AMX_ERR_SLEEP), I've devised the following:

PHP Code:
#include "SDK\amx\amx.h"
#include "SDK\plugincommon.h"

typedef void(*logprintf_t)(charformat, ...);

logprintf_t logprintf;
extern void *pAMXFunctions;

int AMXAPI DebugHandler(AMX *amx)
{
    
logprintf("Shh bby it's okay");
    return 
AMX_ERR_SLEEP;
}

cell AMX_NATIVE_CALL HelloWorld(AMXamxcellparams)
{
    
amx_SetDebugHook(amxDebugHandler);
    return 
1;
}

PLUGIN_EXPORT unsigned int PLUGIN_CALL Supports()
{
    return 
SUPPORTS_VERSION SUPPORTS_AMX_NATIVES;
}

PLUGIN_EXPORT bool PLUGIN_CALL Load(void **ppData)
{
    
pAMXFunctions ppData[PLUGIN_DATA_AMX_EXPORTS];
    
logprintf = (logprintf_t)ppData[PLUGIN_DATA_LOGPRINTF];

    
logprintf(" * Test plugin was loaded.");
    return 
true;
}

PLUGIN_EXPORT void PLUGIN_CALL Unload()
{
    
logprintf(" * Test plugin was unloaded.");
}

AMX_NATIVE_INFO PluginNatives[] =
{
    { 
"HelloWorld"HelloWorld },
    { 
0}
};

PLUGIN_EXPORT int PLUGIN_CALL AmxLoad(AMX *amx)
{
    return 
amx_Register(amxPluginNatives, -1);
}


PLUGIN_EXPORT int PLUGIN_CALL AmxUnload(AMX *amx)
{
    return 
AMX_ERR_NONE;

Code:
#include <a_samp>

native HelloWorld();

main() {
	SetTimer("Booper", 1000, 1);
}

forward Booper();
public Booper()
{
	static i = 0;
	printf("Tick %d %d", gettime(), ++i);
	if (i == 3) {
		printf("Executing debug");
		HelloWorld();
	}
}

Results:
Quote:
[00:18:29] Loading plugin: sandbox
[00:18:29] * Test plugin was loaded.
[00:18:29] Loaded.
[00:18:29] Loaded 5 plugins.

[00:18:30]
[00:18:30] Filterscripts
[00:18:30] ---------------
[00:18:30] Loaded 0 filterscripts.

[00:18:30] Number of vehicle models: 0
[00:18:31] Tick 1515453511 1
[00:18:32] Tick 1515453512 2
[00:18:33] Tick 1515453513 3
[00:18:33] Executing debug
[00:18:34] Shh bby it's okay
[00:18:35] Shh bby it's okay
[00:18:36] Shh bby it's okay
[00:18:37] Shh bby it's okay
[00:18:38] Shh bby it's okay
[00:18:40] Shh bby it's okay
[00:18:41] Shh bby it's okay
My conclusions so far:
1. I think I created compounding DebugHooks (though the function is called SetDebugHook, not AddDebugHook, so maybe just overwriting)
2. Timers are still executed? Are they implemented separately from amx_Exec flow? (#e: Oh, SetTimer is a native function, I followed the wrong train of thought)
3. Code inside timer callback however, is not executed! It seems AMX_ERR_SLEEP works correctly

Now I'll try resuming the amx execution. I've been spoiled with high level languages, so writing a timeout in C(++) is a new experience for me (I know I can't use "sleep" in main thread because it will result in false pausing of the execution and the whole point of checking if debug hook can pause exeuction will be moot)
Misiur is online now   Reply With Quote
Old 09/01/2018, 01:11 PM   #7
Y_Less
Spam Machine
 
Y_Less's Avatar
 
Join Date: Jun 2008
Location: 629 - git.io/Y
Posts: 14,542
Reputation: 2974
Default Re: Loaded AMX viewer

As I said, I'd really suggest starting from the crashdetect plugin. While I know there is a feature for breaking on BREAK (that is why it exists after all), it doesn't quite give the level of control required for either feature - reading memory or single-stepping). Especially not while the code is still encased in the server. Crashdetect replaces AMX execution entirely, making viewing what is going on much simpler.

Southclaws: You sort of just described what I meant by the plugin component. A plugin loaded by the server would control AMX execution, output memory changes, and receive information on breaks. Then what I proposed as a stand-alone application would use those signals to display information, with as I see it you just extending that to allow anyone to read a more well-defined and open set of signals coming from the plugin. I don't think having them internet accessible by default is a good idea, but maybe there is some way to allow them to be routed over an SSH tunnel if desired (that probably would require a very thin console wrapper that you described).

Edit: When I first thought of this, I was just going to make it Windows only for two reasons. 1) I couldn't be bothered with cross-platform GUI concerns, but if the front-end is built one something else that's not a problem. 2) On Windows OpCodes are as they are in pawn-impl.pdf, whereas on Linux they are translated to something entirely unique per machine and run. However again, if we fully take over execution in a custom plugin, the simplest way to deal with that would be to not do any translations, which would also mean that no relocations would be required when telling the front-end about changed memory (hard as it would require knowing exactly when a relocation was required and when it was just data, even in the code segment).
Y_Less is offline   Reply With Quote
Old 17/01/2018, 11:24 PM   #8
Misiur
High-roller
 
Misiur's Avatar
 
Join Date: Jul 2009
Location: Poland
Posts: 2,522
Reputation: 550
Default Re: Loaded AMX viewer

Hello again!
So, I've tried to keep my activity on the project down low, but now I see 7 stars on github so I guess it ain't secret anymore.
Summary of my journey so far, with massive amount of irrelevant info and rambling in a quote block if you want to read, otherwise skip:
Quote:
1. Zeex is a goddamn great craftsman. I copied amx_Exec from amx.c included with crashdetect into the function overwriting standard amx execution flow, made it compatible with crashdetect classes and it worked
2. Then I reread this thread and saw that Y_Less mentioned _W macro. I saw that it is not there, so I checked pawn compiler with fixes (whaddya you know, by Zeex as well), got the amx_Exec from there and replaced what I had.
3. Then I wanted to test if I can read CIP on each loop, so I added printf (logprintf was not available in this class), and I got... weird results? I guess logprintf buffers output, and printf does not, or in reverse, or something completely unrelated. So after quick check, I saw log class which uses logprintf, and now all is well.
4. I wanted opcode names as well, so ******d how to do it best in c++, and they recommended std:tring_view.
5. Which required C++17. My MSVC didn't support it, so I had to update visual studio, which progressed at 1% per 10 minutes... In the meantime I used cmake to create a build set up for mingw, and then it turned out mingw updater had only g++ 6.3 which did not fully support C++17 (string_view to be exact). So I had to download g++ 7 and replace everything, but I downloaded x86_64 version, which caused a lot of linker errors and wasted 20 minutes until I downloaded i684 build. I thought C++17 should be mostly compatible with earlier versions, but it turned out there are quite a few BC breaks. Zeex used std::bind1st which was removed in C++17, so I tried with std::bind but got weird errors about templates having problems with resolving unknown-type, etc. etc. So I had to do a quick overview of lambda functions (weirdly reminding me of javascript closures, but with explicit restrictions on passing local parameters), and did the following change:
Code:
std::bind1st(std::mem_fn(&AMXPathFinder::AddSearchPath), &amx_finder))
// to 
[&amx_finder](auto path) { amx_finder.AddSearchPath(path); }
I think bind creates a wrapper function as well, so I think there's no overhead difference (but it feels stupid just to create a closure only to call another function)
6. And then I added this bit
Code:
LogDebugPrint("Current CIP: %d | Current OP: %s | Param or next OP: %d",
 (uint32_t) cip,
 std::string(AMXOpcodeNames[*cip]).c_str(),
 *(cip + 1)
);
std::this_thread::sleep_for(std::chrono::milliseconds(100));
And now I have


In my opinion, this whole project would take experienced C++ programmer about 20 minutes, but as I said, this is my C++ training ground and done during breaks from work, so don't expect too much.

Kinda fun exploring this weird C++sy world. End of rambling
Now I have questions to people interested in this project, in order of importance:

1. How would you like the plugin to communicate with external programs (for example a GUI I mentioned in first post)? Named pipes, sockets, zeromq, or ******'s protobuf?
Y_Less expressed support for sockets on discord, and I'm leaning towards it as well.
2. I'm wondering if updating the code I copied from amx_Exec is good. This is plugin for amx debugging, but fixing and/or introducing new VM bugs/features (such as now adding "#emit" macro opcodes support) might obstruct ours vision
3. I removed JIT and ASM sections of the VM. Should I restore JIT sections? I think when someone runs a debugger, they don't expect the best available performance option, but I'm certain there are JIT specific runtime bugs.
4. Is the samp's implementaion of amx VM for linux (the one with opcode relocation) relatively bug-free? What I mean by that is - does the relocation introduce any new bugs?
5. I would like to rewrite required by amx_Exec "#define"'s with C++ templates (remember that I want to learn C++ basics with this project) - any objections?
5a. There's an alternative, a nuclear option - reimplementing VM completely in C++, like sourcepawn. I don't lean towards it, but it'd be good to know your opinions.
Misiur is online now   Reply With Quote
Old 17/01/2018, 11:47 PM   #9
[HLF]Southclaw
High-roller
 
[HLF]Southclaw's Avatar
 
Join Date: Apr 2009
Location: England
Posts: 4,769
Reputation: 1293
Default Re: Loaded AMX viewer

Quote:
Originally Posted by Misiur View Post
Hello again!
So, I've tried to keep my activity on the project down low, but now I see 7 stars on github so I guess it ain't secret anymore.
Summary of my journey so far, with massive amount of irrelevant info and rambling in a quote block if you want to read, otherwise skip:


Now I have questions to people interested in this project, in order of importance:

1. How would you like the plugin to communicate with external programs (for example a GUI I mentioned in first post)? Named pipes, sockets, zeromq, or ******'s protobuf?
Y_Less expressed support for sockets on discord, and I'm leaning towards it as well.
2. I'm wondering if updating the code I copied from amx_Exec is good. This is plugin for amx debugging, but fixing and/or introducing new VM bugs/features (such as now adding "#emit" macro opcodes support) might obstruct ours vision
3. I removed JIT and ASM sections of the VM. Should I restore JIT sections? I think when someone runs a debugger, they don't expect the best available performance option, but I'm certain there are JIT specific runtime bugs.
4. Is the samp's implementaion of amx VM for linux (the one with opcode relocation) relatively bug-free? What I mean by that is - does the relocation introduce any new bugs?
5. I would like to rewrite required by amx_Exec "#define"'s with C++ templates (remember that I want to learn C++ basics with this project) - any objections?
5a. There's an alternative, a nuclear option - reimplementing VM completely in C++, like sourcepawn. I don't lean towards it, but it'd be good to know your opinions.
Great progress!

1. I'm leaning more towards protobuf - it solves a lot of problems with trying to implement your own protocols. You'll get type checking out of the box and it will make it much easier to integrate with other languages as you can simply use protoc to generate stubs from a .proto file. Bare sockets would involve a custom built protocol and lots of trial and error to get it interfacing with another app.

2. It could be worth building those things into a separate library then providing another plugin that uses that code to provide fixes. Though we can already do quite a bit with the compiler fork...

3. At first I'd be more inclined to go for no JIT to keep things simpler - but then again it's not something I've ever used.

5a. That would also be interesting, but surely with that route you couldn't debug live server runtime with native calls, right?

Regarding the UI:

I'm reading up a bit more on the vscode debugging protocols and interfaces, I'll have an update with how it might interface but I'm thinking something like this:

HTML Code:
[SA:MP Server] <sdk> [This Plugin] <protobuf/socket/http/etc> [Pawn Debug Adaptor (TS/Go)] <vscode debug protocol> [vscode]
Then the protobuf (or whatever you choose) API can then be connected to another adaptor for another application. I'm not totally sure what other apps provide this kind of debugging tools, maybe big-boy Visual Studio does? Sublime probably has a hacky way of doing it and same with Atom.
__________________



[HLF]Southclaw is offline   Reply With Quote
Old 18/01/2018, 02:48 AM   #10
Crystallize
High-roller
 
Join Date: Aug 2013
Posts: 1,533
Reputation: 305
Default Re: Loaded AMX viewer

What do you want to accomplish with this I don’t get it what would be used for
__________________
ZOMBIE MODE IS BACK UNDER NEW IP
Crystallize is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[Tool/Web/Other] SA:MP Sprite Viewer V1.0 LiamM Tools and Files 4 20/02/2017 09:59 AM
Log viewer for CP TyreseFlex Scripting Help 4 03/01/2017 03:51 AM
[Tool/Web/Other] [APP] Sprite Viewer [v1.0] Rhayziin Lançamentos/Releases 14 09/03/2015 10:04 PM
[Tool/Web/Other] [APP] Sprite Viewer [v1.0] Rhayziin Tools and Files 8 08/03/2015 08:00 PM
GTA SA Viewer Kyle Everything and Nothing 4 02/01/2013 05:02 AM


All times are GMT. The time now is 04:02 PM.


Powered by vBulletin® Version 3.8.6
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.