SA-MP Forums

Go Back   SA-MP Forums > SA-MP Scripting and Plugins > Scripting Help

Reply
 
Thread Tools Display Modes
Old 08/02/2017, 06:58 PM   #1
iLearner
Guest
 
Posts: n/a
Default While loop - extremely slow?

I am using a while loop to search in a array, with strfind... e.g:

PHP Code:
new myarray[MAX_MESSAGES][7][1000] =
{
    {
"hey""Hello, how are"},
    {
"hi""Hello , how are "}
}; 
Command:

PHP Code:
while(count != MAX_MESSAGES)
    {
        new 
Index getIndex();
        
printf("%d",count);
        
count++;
        
printf("%d",Index);
        if(
strfind(myarray[Index][0], texttrue)!=-1)
        {
            
printf("Found it");
            
Index;
            
found true;
            break;
        }    
        else
        {
            
printf("didnt find it");
            
found false;
        }
    } 
Basically that runs the loop until we're at MAX_MESSAGES, but breaks if we found what we're looking for.

I have approx 330 messages (MAX_MESSAGES), using that cmd, it takes around 3-4 second to get response, is it cuz of while, strfind or something else?
  Reply With Quote
Old 08/02/2017, 07:09 PM   #2
NaS
High-roller
 
NaS's Avatar
 
Join Date: Mar 2008
Location: 🇩🇪
Posts: 1,785
Reputation: 635
Default Re: While loop - extremely slow?

Quote:
Originally Posted by iLearner View Post
I am using a while loop to search in a array, with strfind... e.g:

PHP Code:
new myarray[MAX_MESSAGES][7][1000] =
{
    {
"hey""Hello, how are"},
    {
"hi""Hello , how are "}
}; 
Command:

PHP Code:
while(count != MAX_MESSAGES)
    {
        new 
Index getIndex();
        
printf("%d",count);
        
count++;
        
printf("%d",Index);
        if(
strfind(myarray[Index][0], texttrue)!=-1)
        {
            
printf("Found it");
            
Index;
            
found true;
            break;
        }    
        else
        {
            
printf("didnt find it");
            
found false;
        }
    } 
Basically that runs the loop until we're at MAX_MESSAGES, but breaks if we found what we're looking for.

I have approx 330 messages (MAX_MESSAGES), using that cmd, it takes around 3-4 second to get response, is it cuz of while, strfind or something else?
I hope you tested without the prints. If you didn't, try without since print is the slowest PAWN function I know - relative to what it does of course.

Also strfind can be quite slow, it's just another loop going through the whole content of the strings, depending on the search term it has to check every character, and sometimes even several characters multiple times.

It seems to be a bit slow, but considering you have 330 messages (probably lengthier ones as well) it's not going to speed up much by optimization.

Btw what do you need the getIndex function for? You could just use a for-loop for indexes. Or use count as index in the existing code. That's not going to improve much but still removes the overhead of having to get an index from a function, although it's already existing.
NaS is offline   Reply With Quote
Old 08/02/2017, 07:36 PM   #3
iLearner
Guest
 
Posts: n/a
Default Re: While loop - extremely slow?

The 2 functions i use:
PHP Code:
new Indexes[MAX_MESSAGES];
stock FreeIndexes()
{
    for(new 
=0iMAX_MESSAGESi++)
    {
        
Indexes[i] = 0;
    }
}
stock getIndex()
{
    new 
Index random(MAX_MESSAGES-1), j;
    for(new 
=0iMAX_MESSAGESi++)
    {
        if(
Indexes[i] != Index)
        {
            if(
Indexes[i] == 0)
            {
                
Indexes[i] = Index;
                break;
            }
             
Index;
        }
    }
    return 
j;

FreeIndses() is called on starting of the command.


I cant use the normal for loop for the only reason that i want to access random elements within the array, not crescent or decreasing, thus i made the function getIndex which checks if the index hasnt been used already, if not return it & store it into the indexes array... until its full...

I couldnt think of another way... i know i am using alot of loops... but is there any other way?
Response is kinda 4-5 seconds.
  Reply With Quote
Old 08/02/2017, 09:35 PM   #4
NaS
High-roller
 
NaS's Avatar
 
Join Date: Mar 2008
Location: 🇩🇪
Posts: 1,785
Reputation: 635
Default Re: While loop - extremely slow?

Quote:
Originally Posted by iLearner View Post
[...]

I cant use the normal for loop for the only reason that i want to access random elements within the array, not crescent or decreasing, thus i made the function getIndex which checks if the index hasnt been used already, if not return it & store it into the indexes array... until its full...

I couldnt think of another way... i know i am using alot of loops... but is there any other way?
Response is kinda 4-5 seconds.
I don't see the reason for accessing them randomly, but you can just shuffle the Index Array. After all this is the same as you do now, but you can do it before executing the main code and save time for the actual calculation. That removes at least one loop from within the main loop.

Also, can you lower the string length? 1000 does not sound reasonable at all. If that's the length of strings you use we don't need to continue trying to make it faster, because that's just how slow it would be!

Otherwise there is not much you can do.
It would be way easier to optimize if you checked for exact matches. That allows for hashing and based on that, other ways to optimize.

I'd suggest you use (my)SQL. PAWN is just bad for matching strings at this scale.

Just think about how many checks are involved. It compares character for character, byte for byte and each byte is read from a global, multi-dimensional array. Even getting one byte involves further calculations because of the structure. Sounds neglible but consdering you have a few million characters, it's not. That's basically the worst case scenario.
NaS is offline   Reply With Quote
Old 08/02/2017, 10:00 PM   #5
iLearner
Guest
 
Posts: n/a
Default Re: While loop - extremely slow?

Quote:
I'd suggest you use (my)SQL. PAWN is just bad for matching strings at this scale.
Could you elaborate a bit?
  Reply With Quote
Old 08/02/2017, 11:25 PM   #6
NaS
High-roller
 
NaS's Avatar
 
Join Date: Mar 2008
Location: 🇩🇪
Posts: 1,785
Reputation: 635
Default Re: While loop - extremely slow?

Quote:
Originally Posted by iLearner View Post
Could you elaborate a bit?
From CompuPhase's web page:

Quote:
Unlike many languages, pawn is not intended to write complete full-scale applications in. pawn's purpose is to script the functionality provided by an application or by a device.
PAWN is a quite minimalistic language which is meant for small (embedded) systems.
It's bascially designed to coordinate hardware and provide the natives and functionalities of the host program to an exchangable script. Just like it's implemented in SAMP, the server does the actual work (natives, syncing players, network..) and PAWN is used to script it.

What you are trying to do is basically not what it was made for.
PAWN is really fast as such, and perfect for what it was made. But going through a few hundred thousands of cells and then passing them to the server for further checks just takes its time.

(My)SQL should be what you are looking for. You can dynamically add/remove as many data sets as you like, query for matches within milliseconds and all this without the need to recompile a 30MB amx file.

I think you can speed up the code you wrote, BUT not to a degree it wouldn't hang your server notably!
NaS is offline   Reply With Quote
Old 09/02/2017, 01:02 AM   #7
Lordzy
High-roller
 
Lordzy's Avatar
 
Join Date: Mar 2012
Location: NetherRealm
Posts: 2,698
Reputation: 1177
Default Re: While loop - extremely slow?

The more data you add in, the more slower it goes. I suggest you to consider what NaS has stated, you can use database management system to fetch your result in few milli-seconds which is ALOT faster than to fetch a string from a 3D array in PAWN. Also, if you're ever building something such as what you're doing, you should have a look over packed strings.
__________________
Currently inactive - I don't play at any SA-MP servers nor work on anything in PAWN for now. The projects that I've done so far in PAWN, which requires updates will be taking some time.
Lordzy is offline   Reply With Quote
Old 09/02/2017, 05:24 PM   #8
iLearner
Guest
 
Posts: n/a
Default Re: While loop - extremely slow?

Suppose i have data stored in MySQL / SQL as following:

IDQuestionAnswer0Answer1Answer2Answer3
0HelloHey there!Sup buddyYoHow are you
1...............

How can i do something like, go through all of them, and check (i do with strfind) if the parts of given text as input matches somewhere in those records, if yes, select 1 of the answers randomly.

Any idea?
  Reply With Quote
Old 09/02/2017, 07:16 PM   #9
NaS
High-roller
 
NaS's Avatar
 
Join Date: Mar 2008
Location: 🇩🇪
Posts: 1,785
Reputation: 635
Default Re: While loop - extremely slow?

Quote:
Originally Posted by iLearner View Post
Suppose i have data stored in MySQL / SQL as following:

How can i do something like, go through all of them, and check (i do with strfind) if the parts of given text as input matches somewhere in those records, if yes, select 1 of the answers randomly.

Any idea?
You don't even need strfind or anything like it.

Code:
"SELECT * FROM table WHERE Answer0 LIKE '%%searchterm%%' OR Answer1 LIKE '%%searchterm%%' (...)"
Will return all rows (including all fields for all rows) that have a match anywhere in the text (searchterm).
You can then just pick one of the fields if one or more rows have been fetched.

AFAIK this only works for MySQL, SQL requires a bit more than that.
If the searchterm is input from a user, you may want to escape it to avoid injection.

To ignore case sensitivity replace each '%%searchterm%%' with LOWER('%%searchterm%%').

PS: If you were going to use strfind, you would've eliminated the point of using MySQL for this. It would be even slower!
NaS is offline   Reply With Quote
Old 09/02/2017, 10:02 PM   #10
iLearner
Guest
 
Posts: n/a
Default Re: While loop - extremely slow?

Hell it freezes the server until its doing the thing... which is now 2-3 seconds.

And no, the messages aint long 1000, the longest one is 180, and most of them are 30-50... i changed the size from 1000 to 180... but it freezes the server for few seconds and sometimes freezes it at all.

Any other way to obtain the thing? using any other thing ? (eg calculations etc on another server / web server) any thing at all?
  Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Vehicle loop too slow ilepopivanov Scripting Help 2 09/06/2015 04:46 PM
Loading servers makes the client extremely slow in responding Biesmen Client Support 5 07/02/2013 02:54 PM
timer will be slow and slow? peterory Scripting Help 0 02/01/2013 01:02 PM
The Horribly Slow Murderer with the Extremely Inefficient Weapon Danny Everything and Nothing 9 02/11/2010 10:49 AM
Slow script or slow streamer? ruckfules99 Help Archive 5 31/03/2010 08:08 PM


All times are GMT. The time now is 10:42 AM.


Powered by vBulletin® Version 3.8.6
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.