Académique Documents
Professionnel Documents
Culture Documents
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
SomeUrgentlyNeededChanges
toMakeBitTorrentClients(andProtocol)Function
inTheirUser'sBestInterests
Abstract
ThesearechangesthatprimarilyneedtobemadeinBitTorrentclient
programuserinterfaces,butsecondarilyneedtobemadeinthewaythe
clientprograminteractswiththecomputersystemitisrunningon.
Therearesomesuggestionsforsomeprotocolchanges,butthesechanges
donotaffectthewaythefilesthemselvesaretransferred.
TheBitTorrentprotocolshould(inthelongrun)beoverseenbythe
InternationalTelecommunicationsUnion(ITU),inthecapacityofproviding"a
frameworkfordocumentingtheprotocolstatemachines"(ClienttoClient,
ClienttoTracker&TorrentFileSyntax).
TheITUshouldinnowayinterferewiththeevolutionofdistributedfile
transferprotocols,butassistwherepossibletomakethesenetworksas
compatibleandinteroperablewithIPandNonIPnetworksaspossible.
Preface
Overall,thelongtermtrendintheevolutionoftheBitTorrentprotocolistowardshigherlevelsofsecurityatalllayersof
theprotocol'sfunctionality.However,everyaspectofBitTorrent'suseisaffectedbyongoingsecurityconcernsand
problems.
Notably:theeraofblindlytrustingBitTorrentclientsishistory
BTClientverificationisabouttheonlywaytospotwebrobotsapplicationspretendingtobeactualBTclients.
FakeBTclientsmostlyarewebrobotswhosegoalistotrackdownpeopleforabusebecauseofsupposed
copyrightissues.
ThefakeBTclientpolicywillprobablyevolveintoverifyingBTclientsanbanningthefakeones,andtheIP
addressesanddomainstheyuse.
TheremaypossiblyevolvearealtimepublicdatabaseofreverseIPs(andapplicationfingerprints)fordangerous
BTclients.
BTClientVerificationandBTProtocolEncryptionarepartofabiggerpictureofprotocolandapplicationchangefor
securityorientedreasons.BitTorrentprotocolandsystemsarebeginningtointersectwithOnionRoutingprotocolsand
systems,outofpracticalityfortheirusers.BitTorrentusersintheWesternsphereareinasmuchdangerasTorusers
http://hireme.geek.nz/btusabilityfixesneeded.html
1/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
areinothernations.
Theneedforreal(butmainlyfunctional)security.Therehastobe"securityintransmission"fromtheOSI"LinkLayer"
intothe"ApplicationLayer"foraBitTorrentusertobesafefromtheextremecivilrightsabusesthathavetakenplacedue
tothepretextofcopyrightlawbeingwrongfullyappliedtobinaryobjectsasopposedtophysicalobjects.
MultiplelevelsofBTencryptionareessentiallymandatoryasof2014,butonlysinglelayerencryptionisavailable.Using
theexistingsinglelayerencryptionhaspracticallybeenmandatorysince2009.Nootheroutboundtransmissionoptionis
safe,yetBitTorrentclientapplicationsfailtowarnapersonofinsecuretransmissionorusabilitysettings.
ManyInternetusershaveWifimodems(and'freeaccess'localnetworks)intheirhomesandflats.ManyInternetusers
runToraswell,andVirtualPrivateNetworking(inameshorweb)isevolvingintoafunctionalityofmanyInternet
applications.ToassociateanIPaddresswithaphysicaladdressandthebehaviorofpeopleatthatphysicaladdressis
legallynebulous.Thereissimplytoomuchroomforabuseandcorruptioninprivatesectorandgovernmentinthisarea.
BitTorrentRealpolitik
CopyrightLawintheEnglishspeakingworldhasbeenturnedintoasortoftyrannical"jointventure"State&Copyright
Sectorpropertyconfiscationlaw.Thestateendsupbeingthebiggestloserofall,butthegovernmentsintheEnglish
speakingworldaremoreorlessenslavedtovaryingdegreestothissocalledCopyrightMafia.Thelongtermequationof
howthecopyrightlawshaveevolvedisnotgood,andindividuals(butnotcorporations)thatcreatecopyrightablecontent
arenowthegreatestvictimsofthecopyrightlawsinplace.
ThereisalegaltacticofentrapmentusedespeciallyinheUS(andCanada,byproxy)wherethecopyrightholdergetsall
theperson'smoneyandthegovernmentisshackledwithprisonandcourtcosts(andthepersonbeingrendered
nearlypermanentlylegallyunemployable).TheendproductbeingCopyrightHoldergetsasmallamountofmoney,and
thegovernmentlosesmanymillions(butmainlybillions).
TheongoingCanadianexample(FEB2014)
NotethatNewZealandandtheUS,UK,Swedenetc...arefarmorecorruptandabusivewiththeirownsimilar
laws
ACanadianinternetserviceproviderhasbeenorderedtohandoverthenamesandaddressesofabout
2000customerswhoallegedlydownloadedmoviesonline.[...]Asaresult,thoseTekSavvycustomerscould
eventuallyreceivealetterfromVoltagethreateninglegalaction.UnderthefederalCopyrightAct,statutory
damagesfornoncommercialinfringementrangebetween$100and$5,000.[...]
TekSavvy'sLeadCounselNicholasMcHaffie(alawyerwithStikemanElliott)saidthesafeguardsthejudge
putinplacewouldputVoltageona"tightleash"andhelptodiscouragecopyrighttrolling.
"Theconcernisthatingetting2000names,theCopyrightHoldercansimplysendoutabunchoflettersthat
threatenlegalproceedingsandmakeoutrageousdemandsforcompensationandsayyouregoingtoface
alawsuitifyoudont"hesaidinaninterviewwithCBC'sTheLang&OLearyExchange.
VoltagehasbeenaccusedofthreateninglawsuitsagainstthousandsofBitTorrentusersintheUS.Typically
internetusersdonothavetheresourcestofightthethreatofalawsuit,McHaffiesaid.Butthecase
managementprocesswillensurethatVoltagedoesnotoverstateitscaseintheletterstoalleged
downloaders,hesaid.
Therewasarealconcernthatwhatwehadherewasnotanoccasiontoenforcecopyright,butratheran
efforttosendoutalargenumberoflettersandreapabitofarewardusingtheprocessofthecourt.The
courtrespondedbysavingwewanttobeverycarefulbeforeweallowthattohappeninCanada,McHaffie
added.VoltagewasalsoorderedtopayanycoststhatTekSavvyincursinidentifyingthecustomersinthe
case,aswellaslegalfees.[...]CPPICDirectorDavidFewersaidhisreadofthedecisionisthatthecourt
wouldnotbeeagertoassignpenaltiesatthehigherrangeofwhattheCopyrightActallows.
"IfVoltageisaskingforfiguresinexcessof[$100]Ithinkthecourtisgoingtoshutthemdownprettyquickly"
Fewersaid.
"Andifthat'sthecaseIthinkVoltageisdonebecausethisisnolongeraviablebusinessmodel.Andthat's
whatthewholecopyrighttrollthingisabout,it'saboutusingthecourtprocesstogetsettlementsthatarein
http://hireme.geek.nz/btusabilityfixesneeded.html
2/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
excessofwhatyoucouldgetfor[actual]damagestoscarepeopleintosettling."
FewersaidhewashappythatthecourtwillvetanylettersthatVoltagesendstoallegedcopyrightoffenders,
sincethey'retypicallydesignedtoscarepeopleintosettlingacase.
"Alotofpeoplejustpaythesettlementratherthandealwiththeuncertaintyandtheanxietyoftheclaimand
themodelispredicatedonthat,"hesaid.
"Certainpeopleareriskaverseandit'scheapertosettleratherthantohirealawyertodealwithit,evenif
youareinnocent."
Thestrangestofall,iswhythereiseventhiskindofcorruptionofthelawexistsatall.Ifreliable(butofficiallyunofficial)
sourcesareright,BitTorrentuseistheleastofconcernfornationsliketheUSorCanadaorNewZealand.Ifyouknow
whomtoask,thisstrategicknowledgeisapparentlyavailable.
Itisawidelyknowninthesignalsintelligenceinterceptprofession(andneverclassifiedasanofficialstatesecret
intheEnglishspeakingcountries)thatMOSSADapparentlyhasplacedanumberofTzarBombatypeweapons
forusewhentheUSfinancesystemmeltsdown.AUSbankruptcycouldterminatetheflowofmoneyto
MOSSAD,andthestateitruns.Thustheneedforanatomicwar.Thereisanuclearweaponsarsenalatits
disposal.
TheseTzarBombaweaponswerestrategicallyplacedasfarbackasthelate1990s,andareapparentlyreal.
Theseweaponsystemswillbeused,thereisnoquestionaboutthat.Possibly100millionsinNorthAmericamay
bekilled,even200millionsisnotunreasonable.Ontopofthis,itcouldallhappeninasingleday.Yet,inlightofthe
SnowdenFilesitisbeyondimprobablethatthoseinthesignalsintelligenceprofessionintheUS(orCanada,or
theUKforthatmatter)areevencapableoffindingtheseTzarBombas.Itisbeyondhopelesstoeventhinkthat
theseweaponscanbekeptfrombeingusedwhentheGlobalFinanceCrisistrulydoesmeltdown.
ForTorrentsthatcontainmultiplefiles
1.FilePrioritizationmustbemademandatoryfortorrentsthatcontainmorethanonefile.Perusinganyotherfile
treatmentpolicywillleadto"deadtorrents"beingcreatedthatcannotbeusablebyanyone.
MostBitTorrentclientssupportthecapabilityofFilePrioritization,soverylittlecodewillneedtobechanged.In
somecasesonlythe"Default"settingswillhavetobechangedandthedatatypeofprioritizationlevel.Itis
recommendablethat16bitunsignedintsbeusedtoencodeFilePriority.Therewillbealmostnoimpactonthe
memoryfootprintforthischange.
FilePrioritizationmustbemadeprogram'sdefaultsetting,iftheclientapplicationdefaultsettingisfornofile
prioritization.
ThereasonforFilePrioritizationbeingmandatoryisthatitallowstheusertochoosewhatpartsofamultifile
Torrentdeservemoreurgencyindownloading.
ThisabilitytochoosedownloadingpriorityhelpskeepmultifileTorrentsmoreactiveandlesssubjectto
unavailabilityorincompleteness.
Forfilesthathavebeensuccessfullydownloadedandverified,thefilepriorityshouldberesetto"0".
ThereshouldbenofileuploadpriorityEXCEPTFORCLIENTSTHATSUPPORT"InitialSeeding".
The"rarestpiecehashighestpriority"protocolforcontentavailabilityregimetakecareofthis.
UserInterfaceImpact
GranularityofFilePrioritization
[]Base500(recommended)
[]Base100
[]Base"NumberofFiles"inTorrent(memoryefficient)
[]Givesmallerfileshigherpriority
[]Eachfilemusthaveauniquepriority
http://hireme.geek.nz/btusabilityfixesneeded.html
3/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
[]Completedfiles,setto"Nominal"(###)
Italics:NotmandatorytodisplayinuserinterfaceifforcedtobeTrue.
2.Downloadingthe"FIRSTSECTOR"and"LASTSECTOR"ofaTorrentfilemustbemademandatoryatthefileand
Torrentlevel.
Thisissuewascreatedbymultimediafilesthatcontainvital[oratleastimportantinformation]attheendof
thefile(asalmostallfiles[exceptforsavedpacketstreams,likeDVBtelevisiondatastreams]containvital
informationaboutthefileatthebeginning).
3.Themeaningof"EndgameMode"mustbeuptotheuser,notthedefaultsettingsoftheclient.
Themathsbehind"EndgameMode"hasalwaysbeensomewhatunrealisticforadecade.Atabareminimum
EndgameModeisoutoftouchwithrealitywithrespecttokeepingtorrentsalive.
Thereasonissimple:Torrentshavevariablenumbersofsectorsveryoftenabove1000to10000.
THEREFOREwaitingforthelast1to5sectorstogointoEndgameModeisalmostmeaningless.Onehastothink
intermsof"PercentCompletion"not"RemainingSectors".
AsallBTclientsuniversallytrack"PercentComplete"theprogrammingimpact(andincreaseincomplexity)isnot
great.Thesechangescouldbemadeonmostclientswithonlyanincreaseinthebinarypayloadoftheclientof
4kb,butforcompletelysafecode8kbto12kb.
Innormaloperations:
Itisnot(norhasiteverbeen)thelast1or5sectorsthatneedhighpriorityindownloading.
Asarulethelast3%to5%ofaTorrentneedsEndgamepriority.
This"EndgameMode"MUSTBEappliedtotheleveloftheIndividualFilebyDefault,nottheTorrentasawhole.
However,forsomecompactBTclientsapplyingEndgameModeforthetorrentasawholeisOK...theoutcome
beingthatitissuboptimal.
Thedatabasemonitoringthetorrentshouldnotneedanupdatetoitsstructuretodothis,astherereallyareonly3
statesforatorrenttransmission"Downloading"+"EndgameMode"+"Seeing"...
EndgameModemustalwaysbelocaltoafile,notaTorrent.[UnlessitisasingleTorrentfile.]
Thecostforthischangeintraffic(andwastage)termsisnotgreat,iftheEndgamerequeststatemachineismade
awareandtracksitsgoal.
ThischangeintheEndgame'statemachine'shouldonlyproduce1%to2%wastageonlesspopularTorrents.
Havingapermissiblewastageof1%[persession,peruser]ofaTorrentistolerable,ifitkeepstheTorrentalive.
TheUserInterfacechangeswouldlooklikethis
EndgameMode
Shouldstartat[]95%[]96%[]97%or
[]Whenafilehas[]10to25or[]25to50remainingsectorsor
[]Whenafilehasunder5%to10%sectorsremainingor
[]Whenaminimalnumberofavailablecopiesexistorareavalable
4.DownloadingAlgorithmFlexibility
ThefollowingdoesnotchangeanyBitTorrentdownloadingprotocolsinanyway.Thefollowingismerelyachange
inthebiasoftherarestpiecesalgorithmfortheclientdownloadingatorrent.Thisbiasshouldneverbeappliedto
torrentswithlessthan100sectors,asRarestPieceAvailablefunctionsperfectlyforthesesmalltorrents,evenfor
dialuporWiFiusers.
http://hireme.geek.nz/btusabilityfixesneeded.html
4/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
CurrentlytheBitTorrentdownloadingalgorithmpreferstheRarestPieceAvailable.Althoughmathematically
"RarestPieceAvailable"hasproventobeagoodchoice,whenitcomestodownloadingmulitgegabytetorrents
"RarestPiece"isverypooratbeingauniformspacefiller.UniformSpaceFillingwilluptheoverallsystematic
"signaltonoiseratio"ofatorrent,withrespecttotheavailabilityoftherarersectors.
Provisionally,theCantorSetisrecommendableasaanalternate.TheSmithVolterraCantorSetwouldmake
agoodalternate.
ThecoreRarestPiecedownloadingalgorithmshouldonlybebiasedbytheSpaceFillingAlgorithm,andthisbias
shouldnotexceed66%overallbutshouldbegreaterthan10%tobeeffective.
CantorSet
Inmathematics,theCantorsetisasetofpointslyingonasinglelinesegmentthathasanumberofremarkable
anddeepproperties.
Itwasdiscoveredin1874byHenryJohnStephenSmithandintroducedbyGermanmathematicianGeorgCantor
in1883.
Throughconsiderationofit,Cantorandothershelpedlaythefoundationsofmoderngeneraltopology.Although
Cantorhimselfdefinedthesetinageneral,abstractway,themostcommonmodernconstructionistheCantor
ternaryset,builtbyremovingthemiddlethirdsofalinesegment.
Cantorhimselfonlymentionedtheternaryconstructioninpassing,asanexampleofamoregeneralidea,thatofa
perfectsetthatisnowheredense.
SmithVolterraCantorSet
Inmathematics,theSmithVolterraCantorset(SVC),fatCantorset,orCantorsetisanexampleofasetof
pointsonthereallineRthatisnowheredense(inparticularitcontainsnointervals),yethaspositivemeasure.
TheSmithVolterraCantorsetisnamedafterthemathematiciansHenrySmith,VitoVolterraandGeorgCantor.
LogisticMap
Thelogisticmapisapolynomialmapping(equivalently,recurrencerelation)ofdegree2,oftencitedasan
archetypalexampleofhowcomplex,chaoticbehaviorcanarisefromverysimplenonlineardynamicalequations.
Themapwaspopularizedinaseminal1976paperbythebiologistRobertMay,inpartasadiscretetime
demographicmodelanalogoustothelogisticequationfirstcreatedbyPierreFranoisVerhulst.
ThebifurcationdiagramofaLogisticMapisselfsimilar.Thesameistrueforallothernonchaoticpoints.Thisisan
exampleofthedeepandubiquitousconnectionbetweenchaosandfractals.
UserInterfaceImpact
DownloadingAlgorithm
[]Justuserarestpiece(default)
[]Use1DCantorSet
[]Use1DSmithCantorVolterraSet
[]Use1DLogisticMap
BiasfromRarestPieceAvailable
[]20%[]40%[]60%
IssuesrelatingtoTrackerhandling
1.RemoveallduplicatedTrackersnomattertheirtype,foreachTorrent.
Everytracker"string"shouldbecheckedforuniquenessusingMD5asabaselinehashsum,oraCRClike
CRC40GSM.UsingCRC40GSMisadequateenoughtoavoidabout99.997%ofpossibletheoreticalASCII
(orUTF8)stringcollisions.ThisisonlyifonedoesnotwanttoputtomuchstressonanyMD5()code
section.The'Initial'and'Final'ValuesoftheChecksumareuptotheimplementation.
Iftwoormoretrackershavethesamechecksum(andor)hashsumthentheyareduplicatedandredundant.
http://hireme.geek.nz/btusabilityfixesneeded.html
5/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
DuplicatedTrackersshouldbeautomaticallydeleted.However,theusershouldbetoldoftheduplication
(andor)shouldbegivenachancetogivepermissionforthedeletion.
TheremaybeaTorrentdatabaseimpactifyouwanttoadddatafieldswiththeapplicableCRCsor
hashsums.Thisisreallyaruntimegarbagecollectionissue,andthatnointernalTorrentdatabasechanges
shouldbemade.
2.AuserselectabledeletionpolicyisneededforHTTP,UDPandHTTPStypeTrackersisneeded.
Asarule,youshouldbeabletochoose(byprotocoltype)ifatrackershouldbeautomaticallydeleted.Thereisno
disadvantageintheautomaticdeletionoftrackersasthe"DistributedHashTree"and"PeerExchange"protocolshave
madepublicandprivatetrackerslessimportant.
TheUserInterfacechangeswouldlooklikethis
TrackerManagement
Security
[]TunnelmyDNSrequestsoutsidethis[]network[]country
[]TunnelmyTrackerrequestsoutsidethis[]network[]country
Do[]DNSrequestsfor([]Verifiedand[]Unverified)clientsoutsidemy[]network[]country
Do[]Trackerrequestsfor([]Verifiedand[]Unverified)clientsoutsidemy[]network[]country
Automaticallydelete
[]duplicatedTrackers(recommended)
[]unreachableTrackers(recommended)
[]httpTrackers(recommended)
[]httpsTrackers(partlyrecommended)
[]udpTrackers
[]askuserdeletionpermissionfor[]http[]https[]udp
IssuesrelatingtoIP4&IP6Portuse
1.BitTorrentshouldonlyuseEphemeralPorts(~32000to65000),similartowhatFTPhasbeendoingfordecades.
ThecurrentIP4andIP6"PortUsePolicy"allowsforportstobeusedthatotherapplications(orprotocols)
mightbeusing.Thisawkwardarrangementmakesforanebulousrelationshipbetweentheprotocoland
ISPs(andlocalareaNetworkManagers)thathavetomanageIPportuse.
FTPusesasemistandardizedrangeofephemeralportsforitsfiletransfers,andBitTorrentshoulddothis
too.
2.IPPortsusedshouldberandomlychangedatleastonceevery90minutes,oratminimallyevery20minutes.Thisis
somewhatofaprivacyissue,butmostlythisispreventanyparticularportfrombeingoverused.
UserInterfacechangeswouldlooklikethis
PortUtilization
[30000]PreferredstartPort[Default]
[60000]PreferredendPort[Default]
[]RandomizePortevery
[]30:00[]60:00[]80:00
[]()5minutes
IssuesrelatingtoCryptography
1.Morekindsofcryptographyneedtobemadeavailable.
2.TheabilitytohavedifferentInboundandOutboundcryptography,onaperTorrentbasis.
http://hireme.geek.nz/btusabilityfixesneeded.html
6/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
IssuesrelatingtoDownloading&Uploading"ModeControl"
1.Currently,theBitTorrentclientchoosesbetween"Fast","Medium"or"Slow"basedonasetofmultipleconstraints.
Slowmode,forfiletransferonDSLorCableModemconnectionscanbequiteglacial.Slowmodeformanyhigh
speedconnectionsshouldbeavoidedwherepossible.However,ifyouareusingamodemtodofiletransfer
Slowmodeisalmostthedefaultmode.Slowmodeworksmostreliablyfordialupusers,andwilldosolongintothe
future.Slowmodeshouldbeusedforslowconnections,whereitsbetterErrorCorrectioncapabilitiesarebest
used.
MediumandFastmodesareverysimilartoeachother,buthavemorehighlyabbreviatedErrorCorrectionand
otherfilesegmentdescriptors.ThesemodesmayhaveemergedafterBitTorrentbecamepopularonhighspeed
reliablenetworks.
Thereisno"TestLoopThru"capabilitytodeterminethemostoptimalmethodforfiletransferanyparticularInbound
orOutboundlink.Usersneedtoabletochoosethesettingthatworksbestforthem,butthesettingscanbefound
automatically.
Somefiletransfermodesaremorerecommendedthanothersforsometypesofconnections.Theadditionofthis
featureshouldnotaffectthefiletransferprotocolsatall.Preferably,thetestloopthrushouldbedonewithother
clientsthattheclientprogramisalreadyconnectedto,althoughthereissomemeritforitbeingdonewiththeclient
programwebsite.
2.SomesortofuserpreferenceforJumbofiletransfermodeneedstoexist,asitsuseisuncleartoalmostallusersand
isprobablyunderusedonDSLandCableModemtransmissioncircuits.
UserInterfacechangeswouldlooklikethis
PreferredFileTransfermodes
[]Slow(fordialupusers)
[]Medium(forWiFi&DSL,orgeneraluse)
[]Fast(forgeneralhighspeeduse)
[]UseJumbofiletransfers(whenpossible)sized[]64k[]32k[]16k
[]Domodeloopthrutest[tobeimplemented]
IssuesrelatingtoDHTandDNSservers
DHTandDNSarevitallyimportanttoBitTorrentfiletransferoperations,andstrategicstrangulationpoints.Many"Manin
theMiddle"attacksarepossible,andBitTorrentfiletransfernetworkscaneasilybedisabledorshutdownifattackedin
theseprotocoldomains.
DistributedHashTree(DHT,BitTorrent'sanduTorrent'sexample)
The"BitTorrentDistributedHashTable(DHT)"hasafundamentaldependencyonbeingintroducedtosomenodesthat
arealreadyinthenetwork.Therearemanysourcesofthesenodes.Forinstance,yourclientislikelytosavenodeson
disktoretrythemwhenyoustartbackupagain.AnyBitTorrentpeersarelikelytobeontheDHTaswell,sothoseare
alsotried.However,ifyoujustinstalledaBitTorrentclient,andyoudonthaveanyBitTorrentpeers,youmustrelyona
bootstrapserver.
TheBitTorrent(company)runs<router.bittorrent.com>(onport8991+aspareintheuTorrentdomain)forthepurposeof
boostrappingitsownproductslikeBitTorrentSync,uTorrentandthestandardBitTorrentclient.However,theseDHT
serversinthe2domainsaresubjectto"DomainNameSeizure"andcouldbemadeunreachablealltooquickly.Thisis
nowaytorunaDHTserversystem,astheredundancyisalmostnonexistent.
EachBTprotocolclientneedstohaveapoolofabout16DHTserversaccessibletoit,but64DHTserverswouldbe
moreoptimal
DHT_Primary,DHT_Fallback01,DHT_Fallback02...DHT_Fallback16
http://hireme.geek.nz/btusabilityfixesneeded.html
7/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
Storedintheformat"{ServerDomain:Port}"formatlikethis"router.bittorrent.com:8991"
Using"Port8991"exclusivelyforDHTisabadidea,BTclientsshouldusePorts(60256...6051261512...
61768)asDHTfallback.
SomeBTclientprogramsallowonetoaddDHTservers,butatbestonly1or2.AddingDHTserversisanadvanced
userfeaturethattakesabitofworktodo,ifyouhaveaccesstoorrunalocalDHTserver.
IdeallyentitieslikeBitTorrent(company)shouldmakeadealwithentitieslikethe"ElectronicFreedomFoundation"(not
justintheUS,butinplaceslikeAustraliawheretheyalsoexist)torunlocalDHTserversintheirdomains.Globallythere
needstobeabout256to512"dedicatedupperlevelDHTservers"toservetheneedsoftheBitTorrentprotocoland
otherprotocols(andprotocolnetworks)thatneedtouseDHT.
Entitieslikethe"ElectronicFreedomFoundation"shouldnotbetheonlyoption,asmanyentitieslikeUniversity
(Fraternities,ComputerScienceDepartments,ElectricalEngineeringDepartments)etc...mightbewillingtosetup
DHTserversforlocalcampususe.
HavingestablishedentitiesrunDHTserversisatbestastopgapmeasure,asamethodofredundancyitshould
onlybeabout1/3ofthesolutiontotheglobalDHTserveravailabilityproblem.
Ideally,someBTclientsshouldsupporttheirownadhocDHTprotocolservers.Itwilltaketime,andsomeminor
tweakstotheDHTcommunicationssystemasawhole(ausermayneedtochoosetobindtoaDHTnetwork:
BitTorrent,qbTorrent,etc...)buthundredsofthousandsofDHTserversiswaybetterthan200oreven500
dedicatedcoreservers.
DHTisincreasinglybeingusedforalotofsecurityorientedfunctions,BitTorrent(company)isdevelopingaChat
networkbasedonDHT
See:http://engineering.bittorrent.com/2013/12/19/updateonbittorrentchat/
See:http://labs.bittorrent.com/experiments/bittorrentchat.html
ForDHT"NodeID"enforcementreasons,theDHTprotocolversionandsoftwarecodebaseversionneedtobe
accessibleintheBTclient.Thispracticemustapplytothe"PeerExchange"and"LocalPeerDiscovery"protocolstoo.
NodeIDenforcementiskeyinthesecuringofDHT.WithNodeIDenforcementDHTbecomeshardertoattack
(especiallywithsybils).TheideaisthatwithNodeIDenforcementsybilattacks(whereonemachinepretendsto
bethousandsofnodes)willbecomeifnotimpossibleatleastnotpracticable.Thenewbootstrapserverwillstill
servenodeswithinvalidnodeIDs(infact,legitimatenodesjustjoiningarenotlikelytoknowtheirexternalIPyet
sothisisnotnecessarilyaproblem).However,asaruletheDHTServerwillneither"pingnoradd"thesenodes
tothenodelistbeforehandingthemout.
DHTserverstresswillkeepincreasingasDHTuseincreases.ThewholeDHTnetworkingconcept(andtheBTprotocol
clientsthatuseit)needsoveralltobecomeatleast10timesmoreredundant(oranorderofmagnitude)thanitisinthe
2015s.
WithincreaseduseofDHT,theprotocolwillcomeunderincreasedsecurityattacks.DHTmaybecompromisedas
HTTPTrackerswereinthe2010s.TheDHTserversaBitTorrentclientaccessesmustbeauthentic!
Relatedreading
http://engineering.bittorrent.com/2013/12/19/dhtbootstrapupdate/(summaryoftheBTDHTserver,andsecurity
issues)
http://libtorrent.org/dht_sec.html(theLibTorrentDHTsecurityextensions)
https://github.com/bittorrent/bootstrapdht(DHTBootstrapServer)
UserInterfaceImpact(I)
DistributedHashTreeServers
UpdateDHTserverlisteveryFortnight[]orMonth[]
UpdateDHTserverlistalsoviaPeerExchange[]orLocalPeerDiscovery[]
UpdateDHTserverlistalsoviaVPN[]
DHTServerUsage
[]RandomizeDHTServerEachTime
http://hireme.geek.nz/btusabilityfixesneeded.html
8/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
[]PreferBestPerformingDHTServers
[]PreferNearestLocalDHTServers
[]UseDHTLocalServerat[]orIP[]
UserInterfaceImpact(II)
[TrackersTab]
Name
ProtocolVersion
DHT
2014aa
DHTSEC2014
2014qq
LocalPeerDiscovery
2013ce
PeerExchange
2013gv
Status
UpdateIn
Seeds
Peers
Blocked
Downloaded
{TrackerList}
IssuesrelatingtotheDomainNameSystem(DNS)
Inthecurrentregime,BTclientsgivetheuserachoiceof2DNSservers.ThismayseemlikeenoughDNSservers,but
thesheerlackofredundancyintheDNSsystemsofmostBTclientsmakesthemsubjecttoshutdown(orneartotalloss
ofinteroperability)attheDNSrequestlevel.
Theonlyremedyisforthe10majorplayersintheBTclientbusinesstosetup10dedicatedDNSservernetworksfor
exclusiveusebytheirBTclientprograms.
Theservernetworksshouldbegloballydistributed,withatleastonededicatedDNSserverpermajorcountrythatuses
thatparticularBTclientprogram.
TheDNSsolutionwouldworkbestifthisDNSworkaroundallowedforalloftheDNSserversinalloftheDNSprivate
networkstobesharedreasonablyequally.However,limitingthesekindsofrequeststo10%or20%ofallDNSrequests
wouldprobablybethefairestandmostreasonableDNSpractice.
Exceptforthe"fixed"dedicatedrootDNSserver[primary&backupaddresses],allDNSserverlocationsshouldexpire
afterabout123daysafterdownloading.Theexpirationdatesshouldbespreadoutover2days,andtheclientprograms
shouldfullyupdatetheirDNSlistsevery100to120daysastrafficloadsandusagepermits.
UserInterfaceImpact
DNSRequests
[]Useapoolof[]16or[]32globalDNSservers
[]Use{Clientname}PrivateDNSservers(recommended)
[]PreferLocalPrivate{Clientname}DNSservers
[]LimitotherPrivateDNSserverrequeststo[]10%[]20%
[]LimitPublicDNSserverrequeststo[]20%[]30%
IssuesrelatingtoVPNs(VirtualPrivateNetworks)
VPNuseforBTprotocolusehaspreviously(2000to2014)beenviausingProxies.Everyinternetapplicationhasto
supportproxiestosomeextentorotherduetothevariednetworkstheseprogramsareusedin.
HistoricallyapersonhadtopayaVPNISPforaccesstotheirVPNnetwork.
Thewholearrangementwouldevolveintoaterribleoneforbothparties.
VPNIPSsarebeinggloballysubjectedtolegalloggingrequirements,forvariedsecurityreasons.WithlegalVPNlogging
requirements,trafficprivacyvanishes.
The'centralplumbing'arrangementofthe'payasyougoVPN'isbecomingtoodangerousforanykindofBTprotocol
http://hireme.geek.nz/btusabilityfixesneeded.html
9/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
use.Thereisnoguaranteeofanyrealprivacywitha'payperuse'VPN,asthisseemstohavebecomehistoryasof
2012.
Hopeisnotlost.VPNscanbeconfiguredinamyriadofdifferentways.
WhataVPNis,muchlesshowoneissupposedtobestructured(orfunction)arestillongoingdebates.Thereare
hundredsofwaystoconfigurehowaVPNoperates,andADHOCVPNsareprobablyabetterlongtermsolution.When
theBTprotocolfirstinstitutedencryption,italmosthadtheeffectofcreatingapointtopointVPN.However,better
solutionsareneededaseventhecurrentencryptionsolutionsusedbytheBTprotocolareshowingtheirage.
uTorrentisoneofthefirstBTclientstogotherouteofusingaVPN,thereareothersthathaveusedothersimilar
solutions.
DuetotherelativeobscurityoftheupdateofVPNfunctionality,thereisnouserinterfacetotheVPNsubsystemto
controlit.
TheslownessoftheuptakeoftheVPNfunctionalityisatestamenttothesolidnessofthecurrentBTprotocolencryption
practices.Yet,duetothesheervolumeofBTtrafficthecurrentBTencryptionpracticesareandalwayshavebeen
subjecttoattacksthatcouldleadtoalossoffunctionality.
However,ifaVPN'ssettingsarenotaccessibleitcreatesrisksequivalenttothecurrentBTprotocolencryption
settingsnotbeingaccessible.
ThereareactuallyalotofVPNprotocolmethodsinuse.SothereisnoshortageofprotocolstouseforaVPN,soitis
advisablethatmorethanonekindofVPNshouldbeused(atthesametime).Overall,inordertoproceedaheadwitha
VPNforBTprotocoluse2layersoftunnelencryptionshouldbeused,onefortheoutbounddatastreamandoneforthe
encryptedoutbounddatastream.
ItisanideaworthconsideringcreatingapsudoIP6network(withintheadhocVPNnetworks)toobscuretheIP4or
IP6sourcesanddestinations.
BitTorrentADHOCVPNtypesthatshouldbesupported
PointtoEndpoint(easiesttodo,optimalformanynetworksandnetworkingconditions)
PointtoEndpointMultihop(moresecure,butstillapointtopointconnectionbutwithintermediatenodes)
PointtoMeshtoEndpoint(evenmoresecure,subjecttostatefulroutingissuessecuritylevelcanvary)
TheprimaryadhocVPNconfigurationissuesare
limitingthenumberofVPNlinks
networkkeepalivesignalling
networkrekeyingintervals
network"dummytraffic"insertion(atallencryptionlayersinvolved)
networkmanagementof2layersofencryption
networkmanagementoftrafficforwarding
networkmanagementofIPaddressremapping
networkmanagementofDNSrequests(notmandatory,ifsetelsewhere)
Issuesrelatingtoclientinitalization(VPNs)
Becauseitisbecomingmore"strategicallynecessary"toattachtoaVPNbeforesendinganyBTtraffic,thefundamental
methodsofhowaBTclientattachestoanetworkwillprobablyhavetogothrusomeevolutionarychange.Thecurrent
networkingattachmentmodelsdoposemanyprivacyandsecurityrisks.
AninitialmodelfornetworkattachmentforBTclientscapableofVPNcapability
1. AttachtoDNSServers
2. AttachtoDHTServers
3. Initialize"PeerExchange"AND"LocalPeerDiscovery"
4. GetDHTNodes
5. AttachtoDHTServerforVPNs
6. InitializelimitedVPN"DebugLogging"
http://hireme.geek.nz/btusabilityfixesneeded.html
10/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
7. GetVPNNodes
8. InitializeVPNNodes,SingleLayerEncryption
9. InitializeVPNNodes,DoubleLayerEncryption
10. InitializeVPNNodes,PassThruNetworking(withorwithoutextraencryption)
11. InitializeVPNvsNonVPNlinkmanagementsubsystem(VPN"keepalive"timers,AddorDropNodesSupervisor,
VPNPolicyManager...)
12. Resumenormalnetworkoperation(VPNs,openIP4orIP6)
UserInterfaceImpactI
AdHocVPNSettings
SendKeepAliveevery[]15or[]30secondswith[]5%variation
Rekeyevery[]15or[]30minuteswith[]5%or[]10%variation
Insert[]1%[]2%dummytrafficintooutgoing[]traffic&[]encryptedtraffic
[]ForbidSymmetricalKeys(recommended)
LimitOutboundVPNlinksto[]5(dialup)or[]10(dsl,WiFi)or[]20(cable)
UserInterfaceImpactII(VPNTab)
[AdHocVPNs]
VPN
Type
Time
up
Connected
Nodes
Dummy
Traffic
Bad
Packets
Rekey
Events
Psudo
wire
22m
413kb
4852kb
37kb
24kb
1.25%
Tinc
31m
10
1572kb
7243kb
63kb
44kb
1.99%
...
...
...
...
...
...
...
...
...
...
UserInterfaceImpactIII(ControlPanel:ProtocolEncryptionSettings)
Someprogramshavealreadytakenalternateapproachestothesesuggestions,sothereisroomforflexibility.
TriblernominallyusesadhocVPNs.
http://hireme.geek.nz/btusabilityfixesneeded.html
11/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
ProtocolEncryption
AllowIncomingLegacyConnections[]
OutgoingTrafficshould
[]ForceEncryption(increasesoverallsecurity,butsomeBTclientsmaynotbereachable)
[]PreferEncryption(recommended)
OutgoingTrafficinaAdHocVPNshouldbe
[]Encrypted(recommended)with[]AES128or[]AES256
[]Clear
Italics:Optional
Issuesrelatingtorouters
CurrentlyusersdonotknowhowmanyhopsawayanyparticularSeedorPeeris.Thereisanongoingdevelopmentofa
nearestpeerdiscoveryprotocol,butitsdeploymentisslowbutongoing.
EverBTclientshouldruninIP4andIP6(wherepossible)aTraceroutetodeterminehowfarawayinrouterhopsa
SeedorPeeris.However,thisisUserInterfaceissuesoiftheuserisnotchoosingtodisplaythisontheSeeds
/PeerspropertieslisttheTracerouteshouldnotbedone.
IP4deliversasolid(butvaryingovertime)hopcount,thatisthenumberofroutersthepackethashadto
transverse.
IP6doesnotcounthopsinitsTraceroute,sothetimedelayortimedispersion(inms)shouldbeused.
NTP,inamodifiedlowcomplexityform(sayNTPBT)couldbeusedtofindpreciselythetimedelaysand
dispersionsofaconnection.
UserInterfaceexample,attheareawhereSeeds&Peersaredisplayed...
IP
http://hireme.geek.nz/btusabilityfixesneeded.html
Port
Client
Hops
Socket
Down
Up
12/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
(ms)
home.test.ca[uTP]
42155 uTorrent
3.0
15
(120)
State
Speed
Speed
20KiB
21.5 clear
22KiB
waka.org.nz
38251 QbTorrent 26
1.1
(211)
5KiB
nulldev.za[uTP]
60123 LibTorrent 28
4.0
(207)
80.0 clear
11KiB
20KiB
IssuesrelatingtoErrorCorrection
CurrentlyitisnotpossibletochoosethekindofancillaryErrorCorrectionBitTorrentuses.ErrorCorrectionissuesare
uptothenetwork,butthereisnoguaranteethatinthefutureBitTorrentwillberunningonnetworksasreliableasthose
today.
ADDINGERRORCORRECTIONMUSTHAVENOIMPACTONTHEFILESECTORTRANSFERPROTOCOLS.
ERRORCORRECTIONMUSTTAKEPLACEINTHEFILESECTORPREFORMATTER(AFTERTHE
ENCRYPTIONBLOCKIFAPPLICABLE).
TheuseandsettingofthepreferredErrorCorrectionshouldbepossiblebecausesomepeoplewillendup
singBitTorrentoververynoisyorerrorpronecircuits.TheBitTorrentfiletransferprotocolshouldnotbe
alteredinanyway,butsomeancillarysignallingmechanismsneedstobecreatedtocopewiththeissue.It
mustbeuptotheprogrammersthatmaintaintheBTprotocolstoaddtheErrorCorrectioncode.
Thefollowingstatediagrammakestheassumptionthatthefinalfiletransfermessageformatterbeforethe
[Transmitter]>{IPInternet}stepisfixedandinvariantinitsbehaviourandthuscanbeignored(andtreated
aspartofthe[Transmitter]).
Thecurrentpipelineforthetransmissionofafilesegmentis
CLEAR
1. [SectorAA]>[Transmitter]>{IPInternet}
2. {IPInternet}>[Receiver]>[SectorAA]
CRYPTO
1. [SectorBB]>[Encrypt"MK::BB"]>[Transmitter]>{IPInternet}
2. {IPInternet}>[Receiver]>[Decrypt"MK::BB"]>[SectorBB]
3. "MK"justmeansMethod+Key.
4. The"::"isjustanoperatorstatingthatthevariablefollowingitgetsoperatedon.
ThepipelineforthetransmissionofafilesegmentwithErrorCorrectionadded:
CLEARwithECC
1. [SectorAA]>[ECCFormatter]>[Transmitter]>{IPInternet}
2. {IPInternet}>[Receiver]>[ECCDecode]>[SectorAA]
CRYPTOwithECC
1. [SectorBB]>[Crypto"MBB"]>[ECCFormatter]>{IPInternet}
2. {IPInternet}>[Receiver]>[ECCDecode]>[Decrypt"MBB"]>[SectorBB]
Inthisstatemachinediagram,notmuchchange.OnlytheECCFormatterandECCDecoderareaddedto
thetransmissionchain.BothshouldbeseparateentitiesfromtheCryptographyorStandardFileSegment
http://hireme.geek.nz/btusabilityfixesneeded.html
13/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
messageformatters.
UserInterfaceexample
PreferredErrorCorrection
[]None.Useexistinglegacymethods.
[]RandomizeECCmethods
[]VoyagerECC[]CassiniECC
[]GalileoECC[]TurboCodeECC
[]LowComplexityParityECC
ECCUsePolicy
[]ForceECCoverProxyConnections
[]ForceECCoverVPNs
[]ForceECCfor"SlowMode"transfers
Issuesrelatingtomeasurementunits
Clientprogramshavehandledmeasurementunitdisplayissuesbetterastimehasgoneonbutthereisstillalotofwork
tobedone.
OneclassicalmistakemadebyuTorrent(butitisnotalone)isthatitdisplaysatorrentthatwasnotdownloadedby
theclientashavingaShareRatioofInfinity.Thisiscrazymaths,andabotcheddisplayissue.
AlluTorrent'sdisplaymistakemeansisthatAmericanscan'tdomathsasBitTorrentisanAmericancorporation.
Measurementunitdisplayshouldnotaffectanyoftheclientprogram'sbytetransferaccountingcode.Theclient's
accountingcodeshouldbeaccuratetowithin()16bytes.
UserInterfaceexample(italics,notreallyrequired)
MeasurementUnits(Sizes,Speeds,Dates)
[]Base1Kilobits,Megabits,...
[]Base5Kilobauds,Megabauds...
[]Base16Kilowords,Megawords,...
[]Base1000KiB,MiB,GiB...
[]Base1024KB,MB,GB...
[]Iprefer[]2or[]3or[]4decimalunitsofprecisionforuserinterfacenumbers.
Display[]Fuzzyor[]Exact:Dates&Times
[]UsetheTorrentobjectsizeasdisplaydefault(ifdownloadedbytes~0)
[]Showfriendlyhashes(823F0AD700...)butcopynormally
StateMachineTransferbetweenBitTorrentClients&Trackers
1.Forpartialsectors,thereisnoqualitystatetransferthathaslowbandwidth.
2.ThereneedstobeanonbinaryXMLbased,statelessClienttoClientstatetransfermechanism.Thecurrent
mechanismisamessofbinarybitfieldsthatarebadlydocumentedandoftenbadlyconstructed.
Geneara1Requirements
StandardsCompliance
IfitisdecidedthatanexistingXMLprotocolshouldbeused,abackuprelatedprotocolshouldavailableasan
alternate.
Acceptable:WirelessBinaryXML.
Acceptable:EfficientXMLInterchange.
However,thecompressedtokenmustnotsacrificeanyextensibilityorupgradeability.
http://hireme.geek.nz/btusabilityfixesneeded.html
14/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
Coding
Suchamechanismshouldbelimitedtotransferringlessthan1000bytesatatime,with1200beingthemaximum
limit.
SuchamechanismshouldonlyusePrintableASCII,toreducecomplexity.Header:<?xmlversion="1.0"
encoding="ASCII6"?>
Suchamechanismshouldhaveaheaderprefixofnomorethan10bytesindicatingcompressionstate.
Encryption&ErrorCorrection
SuchamechanismshouldonlytransferClienttoClient'state'inanEncryptedform.
Suchamechanismshouldprotectthestatetransferwithahashsum.MD5shouldbethebaselineminimum,
SHA256preferred.
Suchamechanismshouldputthehashsumofthetransferintheheader.
Compression
SuchamechanismshouldZIPcompresstheStateTransferXMLdatatokeepitunderthe1000bytelimitinIP4.
ToreducebandwidththereneedstobeanoptionofpermittingBase64encodingandZIPcompression,ifreally
complexstatetransferisneeded.
VersionTracking
Suchamechanismshouldprovideaprotocolversionnumberofthelastagreeduponstandardupdate:Example:
"2014.08"
Suchamechanismshouldforceverysubprotocoltohaveaversionnumber.Example:"2014.08"
MessageNumberTracking
Everymessageshouldhaveanumberstringintheformat"RR_YYY_DDD_HH_$$$"
R:Reservedforfutureuse,maybeuseaMod11checkdigit.
Y,D,H:Year,DayofYear,Hour
$$$:"BBB"to"YYY"(13824states)=13824/{60mx60s}~=3.84messagespossiblepersecond.
Theletterstringattheendshouldreseteveryhour.
Everymessageshouldexpire[andbedeleted]after2minutes.
Thereneedstobe12recordtypes,withabout10subtypesforPreferredCommunicationsState
1. ChokingState.Ifaclientisoverloaded,itneedstoindicatewhatTorrentsareaffectedandhow.Upto10
Torrentstatescouldbetransferred.
2. EndgameMode(mustbeunder100bytes)
3. SynchronisationState.TokeepCryptography,ErrorCorrection,Speedandotherfactorssynchronized.
Only1TorrentStateatatime.
4. DataLossorMessageLoss.{MessageNumberLost,Garbled}
5. ClientFriendlyNameand3bitstringsforclienttoclientauthentication.(NameorUTF16Name,Bitstrings[1
23])
6. DiscoveryState(DistributedHashTree,LocalPeerDiscovery,PeerExchange,Experimental,Test)
7. ClientA<>ClientBVerification,maybeforapartialformofSinglePacketVerification.
8. RemoteDNS&Trackerrequest,ProxyProtocol.
9. PrivateTRACKER:LoginorAuthenticationState.
10. PrivateTRACKER:AccountingState.HowmuchofaTorrenthasbeentransferred.Thisistransmittableto
Trackersonly.MustbeEncrypted.
11. PublicorPrivateTRACKER:OtherClientState,howmanyotherpeoplehaveTorrent"X"?Mustbe
Encrypted.
12. PreferredCommunicationsState
UDP,IP6Affinity
ConnectionResetRequest(verificationhashorbitstringssender'scountdowntimer{128...
8}secondsthisistoworkaroundIP4'sanduTP'ssequencenumberingbugs)
CryptographySettings(Capable,Current,Preferred,Fallback)
ErrorCorrection(Capable,Current,Preferred,Fallback)
UserConnectionState(Dialup,WiFi,DSL,Cable,BusinessInternet)
PreferredFileTransferModes(Slow,Medium,Fast,Jumbo,Experimental,Test)
HAVEs(HAVEALLorHAVENONE),withaTorrenthashasaninitialparameter.
HAVEs(BitfieldorPartialHave),withaTorrenthashasaninitialparameter.
http://hireme.geek.nz/btusabilityfixesneeded.html
15/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
PartialSectorTransferState(unsigned16bitintforthesectornumberunsigned4bitquantized
bitfieldofeachpartialsectorupto20sectorstransmittableatatime).TheTorrenthashsumisthe
initialparameter.
Italics,highlyimportant
IssuesrelatingtoupdatingaClient
1.Currently,traditionalHTTPandHTTPSareusedtodownloadandupdateBitTorrentclients.Thisupdatingmechanism
worksverywellexceptfortheproblemofwebsiteavailability,somethingthattypicallysuffersbadlywhenamajorclient
updateisreleased.
AfewclientscanuseFTPorSFTPforclientupdating,butthisisrare.FTPaspartofaclientupdatemechanismis
stillacapabilitythatisnotused.VeryfewBitTorrentclientshaveanysupportforFTPatallunlesstheyareofthe
dedicateddownloadertype.
Althoughitisnotforeseeablethatanyoralloftheseprotocolswillceasebeingusedforupdatingclientsinthe
future,traditionalfiledownloadingtechnologyisnotthemostefficientwayofupdatingaclient.
Datacompressionschemesareessentiallynotusedforupdatingclients.Thishelpsneithertheusernorthe
creatoroftheclientprogram.
ThebandwidthsavinggainsofBDIFFandCORRAGETTEareunknownintheBitTorrenttradewhenitcomesto
updatingtheclientprogram.
GoogleusesCorragettetoupdatetheChromebrowser,andpossiblyGoogleEarth.YetCorragette'susewith
smallerapplicationsisstilllimited.
CorragettealthoughCPUindependentinpracticeitisdeeplytiedtotheX86CPUbinaryarchitectureinits
currentform.ModifyingaclientupdatingmechanismtosupportCorragettehasnotbeendoneyet.Asmost
BitTorrentapplicationsaresmallalreadysoCorragetteisnotconsideredtobethebestfit.
Bdiffisadequateenough(andCPUcodedecoupledenough)tomorethanadequatelyfittheneedsofclient
updatingforminorversionsforthenextdecade.
2.Everyclientshouldhaveaseparateupdaterprogram.WithuTorrent,thisistheexceptionbutthecostiscomplexity
andsometimestheupdaterjustdoesnotwork.
Assumptionsmadeonrolesoftheprograms
1. TheClientProgramcanonlydownloadthe:Updater,TranslationorHelpFiles,UpdatedVersionofthe
Client.
2. TheUpdatercanonlyinstallthetheUpdatedVersionoftheclient,oranyTranslationorHelpfiles.
3. TheUpdatercando"garbagecollection"keepingthefilestructureoftheinstalledprogramfreeofanyno
longernecessaryfiles.
4. TheUpdatershouldonlyrunfornomorethan20minuteseachday,excludinginstallationruns.
TheusualalgorithmforupdatingtheUpdater(generalized):
ClientProgram:IstheUpdaterchanged?
DownloadUpdaterviasamemethodasthemainprogramupdater.
InformuserofnewUpdaterversion.
Onnextrun,runUpdater.
IfthereisnonewClientProgramversion,Quit.[IncasetheUpdaterandNewClientVersionareboth
availableatthesametime.]
IfthereisanewClientProgramorTranslationorHelpfiles,Installthem.
TheusualalgorithmforupdatingtheClientProgram(generalized):
IsthereanewversionoftheClientProgram?
DownloadtheClientProgramviauserchosenmethod.
UseTorrentfiletoverifyNewClientprogram.
http://hireme.geek.nz/btusabilityfixesneeded.html
16/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
InformuserofnewClientProgramversion,offertoInstallit.
CallUpdatertoinstallthenewClientProgram.
QuitafterlaunchingtheUpdater.
3.Everyclientneedstokeep1to3copiesofolderversions(andorBetaversions)sothatitmaybepossibletorevertto
anolderversion.
KeepingmultipleversionsaroundforbackupopensuptheoptionofrevertingtothemostrecentStableVersionifone
isleavingtheAlphaorBetatrack.
Thisprogramupdatingflexibilityisreallyneededfortimeswhenthecurrentversionisnotworking.
Suggesteddirectorystructure
.../{Torrentclientname}/{Clientdirectory01},
.../{Torrentclientname}/{Clientdirectory02},
.../{Torrentclientname}/{Clientdirectory03}...
Broadly,thewaytheChromeBrowserhasitsdirectorystructurehasthemostmeritsasthisdirectory
structuresupportskeepingpreviouscopiesoftheapplicationavailable.
TheChromeBrowserdirectorystructuremakes"BDIFFandCorragette"updatingmethodspossible
minimizingbandwidth.
Suggestedupdateprocess
Within{Torrentclientname}/update/
Within{Torrentclientname}/temp/or/tmp/soastohaveatemporaryworkingareafortheupdating
process.
Havingthetemporarydirectoryinsidethe/update/directoryisalsoviable.
Thefilesinthetemporarydirectoryshouldbedeleted2to5daysaftertheupdate.
Updates
Updatetothe[]Beta[]Stableversionwhenpossible.
InBeta[]exitmetoStableversionwhenconvenient.
BetatestingCaptcha()[]{Submitbutton}
Use[]ftp[]sftp[]http[]https[]BitTorrenttodownloadupdate.
Donotuse[]sftp[]https[]BitTorrenttodownloadupdate.
Use[]BDIFFforminorversionupdates.
Use[]Corragetteforminorversionupdates.
Updatetheupdaterever[]2weeks[]month
Updatehelp&translationfilesevery[]fortnight[]month
Italics:Optional.
Issuesrelatingtooutboundmessages
1.BitTorrentcanbeveryburstyprotocol.Therecanbeliterally'storms'ofoutboundcontrolmessagepacketstosend
andthisresultsinequivalentstormsofthesesamepacketsbeingreceivedbytheclientattheotherend.
Thesecontrolmessagestormscansometimessurpassthecapacityofthecommunicationslinkfortheclient
programattheotherend,resultinginforcedTCPqueuingandgeneralmisbehaviourofthecommunicationslink
betweentheclients.
Thestorminessofthecontrolmessageprotocolscanleadtocontrolmessagelossesandretransmissions,and
thisshouldbeavoided.
Outboundcontrolmessagesarenotsubjecttoratelimiting(orqueueing).
Whenoutboundcontrolmessagesaresubjecttoqueueingorratelimiting:theoutboundrateisessentiallysohigh
astobemeaningless.
http://hireme.geek.nz/btusabilityfixesneeded.html
17/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
Eachcontrolmessageshouldbeallocatednomorethan2048bytesinafixedsizemessagequeuearray.
2.Controlmessagetypes(choking,haveall,havenone,...friendlyname)don'thavepriorities.MostBitTorrentclients
treatalloutboundcontrolmessagesashavingequalpriority.Thisbizarreprotocolhabithasbeeninplacesincethe
protocolwasinvented,anditonlymakesextrabusyworkforroutersandbridges.
Priority[1
...8]
1
ControlMessageType
ChokingState,Endgame Absoluterequirement
Mode
1
2
UDP&IP6Affinity
VPNPreferences
HaveallorHavenone
Havebitfield
Havebitfield4bit
Cryptographysettings
4
4
ForVPNlinknegotiation
Tosortoutseedersorthosestartingat0%or100%
RealTimeDebugging"data ~1kbto~4kbofstateinformationfordebugging
block"
UDPtoavoidTCP"maninthemiddle"Hangupattack
ErrorCorrectionAffinity ForClienttoClientconnectionsforsome
downloaders
2
3
Notes
StandardHave()+Partial_Have()bitfields
4bitquantizedHavebitfield
Beforeorduringanyfiletransfer
Partialhave4bitquantized PartialHavedatabase,8/16/32/256segments32
database
bitint
PreferredFileTransfer
Modes
Beforeorduringanyfiletransferallowsforuseof
FTP(s),HTTP(s)
IP4,uTPConnectionReset Doneabout120sbeforeneeded,tofixuTP&IP4bug
Requests
PauseState
ExtraDHT,DNSservers
(securesig)
ClienttoClient
Verification
...
...
...
...
...
...
FriendlynameUnicode
UTF16
Friendlynamelong
True/FalseWillResumeIn(seconds)
ToallowDHT&DNSserverliststobepropagated
Toshutdowncommunicationswithclientsthat
misrepresentthemselves
Tersenamedecoderscanfillthegap,sonotcritical
ToaddBuildNumber,OS,32vs64bitversion
Protocoltest'A1A'...'Z9Z' Totestaprotocolbeforereleasingit,$#$tolabel
disambiguateprotocols
Userinterfaceimpact
OutboundControlMessageLimits
Nomorethan[]2[]3[]5persecond.
Queuesize[]25[]50messages.
ControlMessagePriority
[]Defaultmodelor[]Testmodel.
ADDENDUM
http://hireme.geek.nz/btusabilityfixesneeded.html
18/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
Withthenewspaceavailableforprotocolmessages,theHAVEALLandHAVENONEmessageneedstoberevised
intoapuzzle.
The"HAVEALL"messageintheoriginalBTprotocolwassubjecttoa"maninthemiddleattack"attheISProuter.
AHAVEALLmessagecangetaClient<>ClientconnectionshutdowninTCPviasendingbothsidesa"TCP
HANGUP"message.
ThenewBTprotocolforHAVE(ALL|NONE)messageshaveonlytemporarilyfixedtheproblem[ofISPmaninthe
middleattacks].
Thisfixissubjecttobeingundoneatanyminute,sothisfixneedsalongtermfix.
ThereisabettersolutiontotheHAVE"ALLorNONE"Protocol
THEFIXIS:sendapuzzlewithaBinary(1,0)orTerinary(1,0,1)outcome.
Thepuzzlemustbesimpleenough(asinLinearAlgebra,orFactoringforPrimeNumbersorevenCryptocurrencyMining
[ifbothparieshaveASICstodoso])tobesolvedin30secondsbya32bitCPUrunningat100MHzwithonecore.
AsmostCPUsrunningBTapplicationsaretypicallyrunningat400MHz(AndroidDevices)to3500MHz(mostmulti
coredesktops)sothisshouldnotproseaproblemattheclient.
However,thesechangeswouldrequireroutersatISPstosendthissessiondataofftoalocalcloudofcomputers.As
thiscloudofcomputerswouldclearlyconsumeelectricity,stafftimeandwagesthischangecouldendthepracticeof
ISPsexplotingthispartoftheBTprotocolsettoforbidseeding.
Thismeans
Puzzle_outcome{}:0(False,HAVENONE)1(True,HAVEALL)1(Ternaryonly,HAVESOME)
Puzzlescouldbeusedtoconvey"HAVESOME"messages,butthetypeofpuzzletodothisisnotclearor
obviousatthetimeofthisproposal.
IssuesrelatedtoRAMCacheSize
ThereisnoreasonforBitTorrentclientstohavediskcachesizesbeyond100megabytes,andintypicaluse32
megabytesisinmanycasesalmosttoomuch.Soforthesakeofotherprogramsrunningonthehostmachine,
minimizingtheRAMCacheisvitallyimportant.
AsmallRAMCachedoesnotforcethehostoperatingsystemintounproductivediskthrashing.
Userinterfaceimpact(addaroundDiskCachingrelatedpartsoftheuserinterface)
[]limitRAMharddrivecachememoryto
[]32[]16[]12megabytes
Issuesrelatingtolongtermconversionto"PolymorphicBitTorrentProtocols"(butnotaffectingEncryption)
Inordertounderstandwhata"PolymorphicProtocolEngine"isandwhatitdoesyoumustsee
Defcon21DefeatingInternetCensorshipwith"Dust,thePolymorphicProtocolEngine"
(http://youtu.be/3z56andRyCY)&(http://youtu.be/jeX7k1wPog)
"Dust"(asitisinthevideo)isnotanrequirement,butalowercomplexitybranchofitisneededforBTprotocol
use.
APolymorphicEngineforFilteringResistantTransportProtocols(https://github.com/blanu/Dust)
"Dust"isalsoat(https://hackage.haskell.org/package/Dust)Researchpaper
(http://freehaven.net/anonbib/cache/wileydust.pdf)
http://hireme.geek.nz/btusabilityfixesneeded.html
19/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
References
OSILayersandtheOSIModel
Internet_protocol_suite(generallyorganizedaroundOSIlayersoffunctionality)
Hierarchical_internetworking_model(thesimplestwaytoviewlayereddigitalcommunicationssystems)
OSI_model(layergeneralizednumber)
PhysicalLayer(I,rawbittransportnotalwaysreliable)
DataLinkLayer(II,typicallyhasframesandgenerallyreliable)
NetworkLayer(III,typicallyhaspackets)
TransportLayer(IV,butinIPnetworkingsomeprotocolsaremixedwiththeNetworkLayer)
InIP,theseOSIlayersareconsideredtobehighlyequivalant:SessionLayer(V),PresentationLayer(VI)
ApplicationLayer(VII,althoughinmanyprotocolcasesnotdifferentiablefromthePresentationLayer)
CRCs&Hashsums
Cyclicredundancycheck
Polynomialrepresentationsofcyclicredundancychecks
DHT(DistributedHashTree)
DistributedHashTable(generaltopic)
MainlineDHT(usedbymostBTclients)
github.com/bittorrent/bootstrapdht(DHTBootstrapServer)
Diff,bsDiff&Corragette
http://www.natehardisty.com/wp/2011/04/10/howgooglechromeupdates/()
Diff(theDiffalgorithmthatisthebasisformostdifferentialupdatealgorithms)
Datadifferencing()
Deltaencoding(alsoknownasbsdiff)
http://www.daemonology.net/bsdiff/()
http://www.chromium.org/developers/designdocuments/softwareupdatescourgette(GoogleChromeuses
Corragette)
Comparisonofdataserializationformats()
XML,plainandcompressed
XML(Thegeneralpurposeflexibleencodingsystemusedonthewebforcomplexandvaribledata.)
ValidcharactersinXML(thisdocumenttypewasintendedtobeprintable)
Wellformed_document(AsXMLdocumentsarestatefulrecords,theymustbewellformed.)
Propertylist(adatastructurethatcanbeconveyedbyXML,althoughthe.torrentfileformatitselfuses
BENCODE)
BinaryXML(aterserandalsononprinatbleformofXML)
WirelessBinaryXML(seealso:http://www.w3.org/TR/wbxml/analternatemethod)
EfficientXMLInterchange(seealso:http://www.w3.org/TR/exi/)
VPNs(VirtualPrivateNetworks)
VirtualPrivateNetwork(generaltopic)
Template:VPN(generaltopictemplate)
IPtunnel(genericallywhataVPNisordoes)
Opportunisticencryption(whataBTprotocolclientVPNshoulddooruseinitsVPN)
MediatedVPN(whataBTprotocolclientVPNshoulddooruseinitsVPN)
MobileVirtualPrivateNetwork(animportantsubtypeofVPN)
Tunnelbroker(vitallyimportantforVPNuseinBTclientapplications)
Pseudowire(anotableVPNtechnology)
http://hireme.geek.nz/btusabilityfixesneeded.html
20/21
1/28/2015
SomeUrgentlyNeededChangestoMakeBitTorrentClientsFunctioninTheirUsersBestInterests
VPNRisks
VPNblocking(acurrentVPNissueinChina)
Deeppacketinspection(ageneralrisktoallInternetprotocols)
StatefulPacketInspection(ageneralrisktoallInternetprotocols)
TCPresetattack(ariskfactorforallVPNsthataretotallydependentonTCP)
IPblocking(aVPNrisk)
VPNTypes
HTTPtunnel(atypeofVPNthatcouldbeused)
ICMPtunnel(acoverttunnel)
FastLocalInternetProtocol(replacesIP'sTCPandUDP,hasthecapabilityofVPNfunctionality)
Tunnelingprotocol(aplaintexttunnellingprotocol)
Tinc(Opensourceprotocol,significantlyused)
1DFractalDownloadingalgorithmalternates
ListoffractalsbyHausdorffDimension(containsindexof1Dtypefractals)
HausdorffDimension(TheHausdorffmedthodofmeasuring1D,2Dand3DFractals)
Logisticmap(1D,butmemoryefficent),CantorSet(oneofthe1st1Dfractalsnoticedbythemathstrade,
reasonablymemoryefficient)
SmithVolterraCantorset(SVCSisreasonablysuitablefordownloadingTorrentswith1000ormoresegments,
mayhavebetterSystemGain.)
GeneralRiskVectors
Maninthebrowser(typicallynotaproblemforBTclientsthatdonotsupportaddonapplicationframeworks)
LegalRiskVectors
Special_301_Report(PeterDrahos,alawprofessorattheCentreforCommercialLawStudiesatQueenMary
UniversityofLondon'sSchoolofLaw,hascalledtheSpecial301Report"apubliclawdevotedtotheserviceof
privatecorporateinterests".ThislawexplicitlyprotectsandactsinfavorofUSintellectualpropertyowners,most
oftenlargecorporations,againstanyforeignnationalpolicyorunofficialactionthatdoesnotconform,eveninits
domesticlegislation,totheUSpositiononinternationalcopyrightandIP.ThreatofactionunderSpecial301has
beenusedtoinsertUStradelobbybackedadvisersintostates'domesticlegislativeprocessinordertoensure
compliancewithUStradenorms.)
Alotofwhat"CopyrightTrolls"doisevenlegal,sothereisthesiteFightCopyrightTrollsfortheUSsystem.
Anewssitedevotedtothelegalproblemsofdistributedfiletransfer:TorrentFreak
BitTorrentsitesmaydecidetofleetotheTorNetwork,toescapelegalsanction...andthePirateBrowsermaybe
awaytoreachthem.
CanadianVPNservicescouldbeforcedtoalertpiratingcustomers
etc
OpenSourceSystems,thewaytheworldisgoing
NetworksvsHierarchieswhichwillwin?NiallFurguson()
OpenSourceRevolutionvs1%theUSCIAview.()
InitialIdeas
Document
Created
LastRevised
Revisionsmade
Version
June2010toDecember
2012
06September
2013
28Febuary
2015
Textupdates
VPNs
0.84kk Revisable,
Updatable
http://hireme.geek.nz/btusabilityfixesneeded.html
Revisionstate
21/21