Solution to the RACE_conf.pl slowdown

From: Gregory Neil Shapiro (gshapiro+opensrs-dev@gshapiro.net)
Date: Fri Feb 02 2001 - 02:27:23 EST


I've been reading a lot of posts about the slowness in processing domains
due to the huge RACE_conf.pl file. I have come up with a much better way
of checking if a character is allowed and done some light testing.
RACE_conf.pl now loads in .034 seconds. Although I offer no guarantees,
just code, this should work fine.

The patch below is edited. In the interest of not sending a 50000+ line
e-mail message, I've edited out the removal of all of the FORBIDDENCHAR and
ALLOWEDCHAR hashes. If you use this code, be sure to completely remove
those hashes.

Index: lib/RACE.pm
===================================================================
RCS file: /usr/local/src/cvsroot/horsey.gshapiro.net/usr/local/www/test.retsiger.com/lib/RACE.pm,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 RACE.pm
--- lib/RACE.pm 2001/01/08 06:18:07 1.1.1.1
+++ lib/RACE.pm 2001/02/02 07:19:33
@@ -301,7 +301,8 @@
                 # return \%result;
                 #}
                 #this is the new way:
- if (! $ALLOWEDCHAR{$str1})
+ #if (! $ALLOWEDCHAR{$str1})
+ if (! RACECheckAllowed($str1))
                 {
                         $result{Error} = "Forbidden Char: " . $str1;
                         return \%result;
@@ -669,6 +670,24 @@
         return 1;
           
         
+}
+
+sub RACECheckAllowed
+{
+ my $hexvalue = shift;
+ my $value = hex $hexvalue;
+ my @keys = keys %ALLOWGAPS;
+
+ for (0 .. @keys)
+ {
+ if ($value >= $keys[$_] &&
+ ($ALLOWGAPS{$keys[$_]} == -1 ||
+ $value < $ALLOWGAPS{$keys[$_]}))
+ {
+ return 0;
+ }
+ }
+ return 1;
 }
 
 1;
Index: lib/RACE_conf.pl
===================================================================
RCS file: /usr/local/src/cvsroot/horsey.gshapiro.net/usr/local/www/test.retsiger.com/lib/RACE_conf.pl,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 RACE_conf.pl
--- lib/RACE_conf.pl 2001/01/08 06:18:07 1.1.1.1
+++ lib/RACE_conf.pl 2001/02/02 07:19:33
@@ -2,7 +2,7 @@
 
 #RACE Configuration
 
-use vars qw(%FORBIDDENCHAR %CASEFOLDING $RACEPrefix %ALLOWEDCHAR);
+use vars qw(%CASEFOLDING $RACEPrefix %ALLOWGAPS);
 use strict;
 
 #
@@ -11,11727 +11,24 @@
 
 $RACEPrefix = "bq--";
 
+%ALLOWGAPS = (
+ 0x0000 => 0x002d,
+ 0x002e => 0x0030,
+ 0x003a => 0x0041,
+ 0x005b => 0x0061,
+ 0x007b => 0x1100,
+ 0x115a => 0x115f,
+ 0x11a3 => 0x11a8,
+ 0x11fa => 0x3041,
+ 0x3095 => 0x3099,
+ 0x309f => 0x30a1,
+ 0x30ff => 0x3400,
+ 0x4db6 => 0x4e00,
+ 0x9fa6 => 0xa000,
+ 0xa48d => 0xac00,
+ 0xd7a4 => -1,
+ );
 
-%FORBIDDENCHAR = (
- "0000" => 1,
- "0001" => 1,
- "0002" => 1,
- "0003" => 1,
- "0004" => 1,
...
- "fff9" => 1,
- "fffa" => 1,
- "fffb" => 1,
- "fffc" => 1,
- "fffd" => 1,
-);
-
 %CASEFOLDING = (
         "0041" => "0061",
         "0042" => "0062",
@@ -12429,40315 +726,4 @@
         "ff38" => "ff58",
         "ff39" => "ff59",
         "ff3a" => "ff5a",
-);
-
-%ALLOWEDCHAR = (
- "002d" => 1,
- "0041" => 1,
- "0042" => 1,
- "0043" => 1,
- "0044" => 1,
....
- "d79f" => 1,
- "d7a0" => 1,
- "d7a1" => 1,
- "d7a2" => 1,
- "d7a3" => 1,
 );



This archive was generated by hypermail 2.1.3 : Tue Oct 19 2004 - 23:36:15 EDT