fix MDMF tests to include proper 'k' graph
[Imported from Trac: page Performance/Sep2011, version 4]
parent
ab1a5dd059
commit
fd48ebb245
|
@ -23,15 +23,10 @@ significantly with larger k.
|
||||||
## MDMF (trunk)
|
## MDMF (trunk)
|
||||||
|
|
||||||
MDMF is fast! Trunk downloads 1MB/10MB/100MB MDMF files at around 4MBps.
|
MDMF is fast! Trunk downloads 1MB/10MB/100MB MDMF files at around 4MBps.
|
||||||
~~The download speed seems unaffected by k (from 1 to 60)~~. Partial
|
Download speed drops linearly with k, k=60 takes roughly 2x more time
|
||||||
reads take the expected amount of time: O(data_read), slightly quantized
|
than k=30. Partial reads take the expected amount of time: O(data_read),
|
||||||
near the 128KiB segment size.
|
slightly quantized near the 128KiB segment size.
|
||||||
|
|
||||||
(update 25-Sep: the test harness appears to have been flawed, and all
|
|
||||||
MDMF files were actually k=3. A new test is in the works to get proper
|
|
||||||
data on how MDMF read speed varies with k).
|
|
||||||
|
|
||||||
* MDMF read versus k, 100MB [MDMF-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/MDMF-100MB-vs-k.png) (timing5.out)
|
|
||||||
* MDMF partial reads, 100MB [MDMF-100MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-100MB-partial.png) (timing6.out)
|
* MDMF partial reads, 100MB [MDMF-100MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-100MB-partial.png) (timing6.out)
|
||||||
* MDMF partial reads, 1MB [MDMF-1MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-1MB-partial.png) (timing6.out)
|
* MDMF partial reads, 1MB [MDMF-1MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-1MB-partial.png) (timing6.out)
|
||||||
|
|
||||||
|
@ -43,14 +38,14 @@ connections. Compared to the old-downloader (in 1.7.1), downloads tend to be
|
||||||
about 3x slower. (note that this delay is probably completely hidden on slow
|
about 3x slower. (note that this delay is probably completely hidden on slow
|
||||||
WAN links, and it's only the fast LAN connections of the atlasperf1 grid that
|
WAN links, and it's only the fast LAN connections of the atlasperf1 grid that
|
||||||
exposes the delay). In addition, both old and new downloaders suffer from a
|
exposes the delay). In addition, both old and new downloaders suffer from a
|
||||||
linear slowdown as k increases. On the new-downloader, k=60 takes roughly 3x
|
linear slowdown as k increases. On the new-downloader, k=60 takes roughly 2x
|
||||||
more time than k=15. Trunk contains a fix for #1268 that might improve speeds
|
more time than k=30. Trunk contains a fix for #1268 that might improve speeds
|
||||||
by 5% compared to 1.8.2. Partial reads take the expected amount of time,
|
by 5% compared to 1.8.2. Partial reads take the expected amount of time,
|
||||||
although the segsize-quantization was nowhere nearly as clear as with MDMF.
|
although the segsize-quantization was nowhere nearly as clear as with MDMF.
|
||||||
|
|
||||||
|
|
||||||
* CHK (1.7.1/1.8.2/trunk) read versus k, 1MB [CHK-1MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-k.png) (t4/t/t3)
|
* CHK (1.7.1/1.8.2/trunk) read versus k, 1MB [CHK-1MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-k.png) (t4/t/t3)
|
||||||
* CHK (1.7.1/1.8.2/trunk) read versus k, 100MB [CHK-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHK-100MB-vs-k.png) (t4/t/t3)
|
* CHK (1.7.1/1.8.2/trunk) and MDMF (trunk) read versus k, 100MB [CHKMDMF-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHKMDMF-100MB-vs-k.png) (t4/t/t3/t8)
|
||||||
|
|
||||||
* CHK (1.8.2) read versus segsize, 1MB [CHK-1MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-segsize.png) (t2)
|
* CHK (1.8.2) read versus segsize, 1MB [CHK-1MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-segsize.png) (t2)
|
||||||
* CHK (1.8.2) read versus segsize, 100MB [CHK-100MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-100MB-vs-segsize.png) (t2)
|
* CHK (1.8.2) read versus segsize, 100MB [CHK-100MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-100MB-vs-segsize.png) (t2)
|
||||||
|
@ -73,17 +68,19 @@ Likely fixes include:
|
||||||
> old-downloader cheated by reading the whole hash tree at once, violating
|
> old-downloader cheated by reading the whole hash tree at once, violating
|
||||||
> the memory footprint goals (requires O(numsegments) memory), but probably
|
> the memory footprint goals (requires O(numsegments) memory), but probably
|
||||||
> tolerable unless the filesize is really large.
|
> tolerable unless the filesize is really large.
|
||||||
|
* note that 1.7.1 "cheats" with both the crypttext_hash_tree and the
|
||||||
|
block_hash_tree. MDMF in trunk cheats with the block_hash_tree and
|
||||||
|
does not have a crypttext_hash_tree
|
||||||
* encourage use of larger segsize for large files (at the expense of
|
* encourage use of larger segsize for large files (at the expense of
|
||||||
> alacrity)
|
> alacrity)
|
||||||
* use unencrypted HTTP for share reads
|
* use unencrypted HTTP for share reads
|
||||||
|
|
||||||
readv() is the least-work/most-promising, ~~since MDMF has readv() and
|
readv() is the least-work/most-promising, since MDMF has readv() and
|
||||||
achieves high speeds. First step is to do whatever MDMF is doing~~
|
is faster than trunk, although not as fast as 1.7.1.
|
||||||
|
|
||||||
## Future Tests
|
## Future Tests
|
||||||
|
|
||||||
* measure alacrity: ask for random single byte, measure elapsed time
|
* measure alacrity: ask for random single byte, measure elapsed time
|
||||||
* measure partial-read speeds for CHK
|
|
||||||
* measure SDMF/MDMF modification times
|
* measure SDMF/MDMF modification times
|
||||||
* measure upload times
|
* measure upload times
|
||||||
* using existing data as a baseline, detect outliers in real-time during the
|
* using existing data as a baseline, detect outliers in real-time during the
|
||||||
|
|
Loading…
Reference in a new issue