fix MDMF tests to include proper 'k' graph
[Imported from Trac: page Performance/Sep2011, version 4]
parent
ab1a5dd059
commit
fd48ebb245
|
@ -23,15 +23,10 @@ significantly with larger k.
|
|||
## MDMF (trunk)
|
||||
|
||||
MDMF is fast! Trunk downloads 1MB/10MB/100MB MDMF files at around 4MBps.
|
||||
~~The download speed seems unaffected by k (from 1 to 60)~~. Partial
|
||||
reads take the expected amount of time: O(data_read), slightly quantized
|
||||
near the 128KiB segment size.
|
||||
Download speed drops linearly with k, k=60 takes roughly 2x more time
|
||||
than k=30. Partial reads take the expected amount of time: O(data_read),
|
||||
slightly quantized near the 128KiB segment size.
|
||||
|
||||
(update 25-Sep: the test harness appears to have been flawed, and all
|
||||
MDMF files were actually k=3. A new test is in the works to get proper
|
||||
data on how MDMF read speed varies with k).
|
||||
|
||||
* MDMF read versus k, 100MB [MDMF-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/MDMF-100MB-vs-k.png) (timing5.out)
|
||||
* MDMF partial reads, 100MB [MDMF-100MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-100MB-partial.png) (timing6.out)
|
||||
* MDMF partial reads, 1MB [MDMF-1MB-partial.png](../raw/attachments/Performance/Sep2011/MDMF-1MB-partial.png) (timing6.out)
|
||||
|
||||
|
@ -43,14 +38,14 @@ connections. Compared to the old-downloader (in 1.7.1), downloads tend to be
|
|||
about 3x slower. (note that this delay is probably completely hidden on slow
|
||||
WAN links, and it's only the fast LAN connections of the atlasperf1 grid that
|
||||
exposes the delay). In addition, both old and new downloaders suffer from a
|
||||
linear slowdown as k increases. On the new-downloader, k=60 takes roughly 3x
|
||||
more time than k=15. Trunk contains a fix for #1268 that might improve speeds
|
||||
linear slowdown as k increases. On the new-downloader, k=60 takes roughly 2x
|
||||
more time than k=30. Trunk contains a fix for #1268 that might improve speeds
|
||||
by 5% compared to 1.8.2. Partial reads take the expected amount of time,
|
||||
although the segsize-quantization was nowhere nearly as clear as with MDMF.
|
||||
|
||||
|
||||
* CHK (1.7.1/1.8.2/trunk) read versus k, 1MB [CHK-1MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-k.png) (t4/t/t3)
|
||||
* CHK (1.7.1/1.8.2/trunk) read versus k, 100MB [CHK-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHK-100MB-vs-k.png) (t4/t/t3)
|
||||
* CHK (1.7.1/1.8.2/trunk) and MDMF (trunk) read versus k, 100MB [CHKMDMF-100MB-vs-k.png](../raw/attachments/Performance/Sep2011/CHKMDMF-100MB-vs-k.png) (t4/t/t3/t8)
|
||||
|
||||
* CHK (1.8.2) read versus segsize, 1MB [CHK-1MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-1MB-vs-segsize.png) (t2)
|
||||
* CHK (1.8.2) read versus segsize, 100MB [CHK-100MB-vs-segsize.png](../raw/attachments/Performance/Sep2011/CHK-100MB-vs-segsize.png) (t2)
|
||||
|
@ -73,17 +68,19 @@ Likely fixes include:
|
|||
> old-downloader cheated by reading the whole hash tree at once, violating
|
||||
> the memory footprint goals (requires O(numsegments) memory), but probably
|
||||
> tolerable unless the filesize is really large.
|
||||
* note that 1.7.1 "cheats" with both the crypttext_hash_tree and the
|
||||
block_hash_tree. MDMF in trunk cheats with the block_hash_tree and
|
||||
does not have a crypttext_hash_tree
|
||||
* encourage use of larger segsize for large files (at the expense of
|
||||
> alacrity)
|
||||
* use unencrypted HTTP for share reads
|
||||
|
||||
readv() is the least-work/most-promising, ~~since MDMF has readv() and
|
||||
achieves high speeds. First step is to do whatever MDMF is doing~~
|
||||
readv() is the least-work/most-promising, since MDMF has readv() and
|
||||
is faster than trunk, although not as fast as 1.7.1.
|
||||
|
||||
## Future Tests
|
||||
|
||||
* measure alacrity: ask for random single byte, measure elapsed time
|
||||
* measure partial-read speeds for CHK
|
||||
* measure SDMF/MDMF modification times
|
||||
* measure upload times
|
||||
* using existing data as a baseline, detect outliers in real-time during the
|
||||
|
|
Loading…
Reference in a new issue