Opened 6 years ago

Closed 5 years ago

#934 closed defect (fixed)

Speed up reseeding by using zip file URL

Reported by: zzz Owned by: Meeh
Priority: minor Milestone: 0.9.14
Component: www/reseed Version: 0.9.5
Keywords: Cc: meeh@…
Parent Tickets: Sensitive: no

Description

  • Speed up reseeding by fetching a single zip file with zipped RIs. Reseed time is dominated by RTT and SSL handshake (700-1300 ms time per fetch). This could save 2 minutes or more. Requires support on the reseed hosts. Standard zip file name relative to index, or configured?
  • Add form and button on /configreseed to fetch and reseed from a zip URL, for those in countries where reseed is blocked.

Note: May require substantial defensive programming when processing zip file contents.

Subtickets

Change History (8)

comment:1 Changed 6 years ago by Meeh

Cc: meeh@… added

comment:2 Changed 6 years ago by Meeh

Owner: changed from zzz to Meeh
Status: newassigned

First of all, the support on the server side should be added. Next in line should be issues in #933 and then continue on the form in /configreseed.

comment:3 Changed 6 years ago by Meeh

Added server support for zip files.
New reseed scripts available at:
http://meeh.i2p/reseed/reseed_mysql_zip.phps (syntax highlighted)
http://meeh.i2p/reseed/reseed_mysql_zip.sphp (download)

Logic:

$zipfile = NETDB_DIR.'/i2pseed_'.md5($remote_ip).'.zip';
chdir(NETDB_DIR);
if (!file_exists($zipfile)) {
    $ok = create_zip($clientRouterList, $zipfile);
    if ($ok) {
        $clientRouterList[] = basename($zipfile);
    }
} else if (filemtime($zipfile)<(time() - 86400)) {
    unlink($zipfile);
    $ok = create_zip($clientRouterList, $zipfile);
    if ($ok) {
        $clientRouterList[] = basename($zipfile);
    }
} else {
    $clientRouterList[] = basename($zipfile);
}

The new feature is added to my reseed hosts.

comment:5 Changed 6 years ago by zzz

The plan is to use the new 0.9.9 su3 file format instead of plain zip. This simultaneously solves the "boot into untrusted network" problem. Su3 handling code in ConsoleUpdateManager?.handleRouterFile() may be used as an example for processing the su3 file.

comment:6 Changed 6 years ago by zzz

Component: router/netdbwww/reseed
Milestone: 0.9.90.9.10

Too late for 0.9.9. Haven't seen any code yet from Meeh. Let's hope for 0.9.10.

comment:7 Changed 5 years ago by zzz

Milestone: 0.9.100.9.14

In d7e72f9e11cd6ba14c7b2be90040f2fd00709dd5 0.9.13-11. Briefly tested. Still need widespread server support, and certs checked in to installer/resources/certificates/reseed/

comment:8 Changed 5 years ago by zzz

Resolution: fixed
Status: assignedclosed

up and working

Note: See TracTickets for help on using tickets.