Download Report

! Review
! Application model
! Presentation layer
! What is P2P?
! How to build large-scale services?
Suguru YAMAGUCHI
Nara Institute of Science and Technology
Graduate School of Information Science
Information Network1
OSI
1
2
! Layers above the Session layer are called upper
layer protocols
TCP/IP
Process Unit
Application
Message / Stream
Transport
Transport
Transport Packet
Port
Network
Internet
IP Datagram
IP address
Data Link
Network Interface
Frame
Datalink Address
Physical
Hardware
Address
–  defined for each application
–  firmly constrained to application requests
Application
Presentation
Information Network1
Session
Information Network1
3
! Defines a semantic unit of communication
4
! Data expression form
–  Transaction
–  Session
–  This layer forms a basis to correctly express data in different
platforms
–  example
! Defines “process” for communication “unit”
•  expression of “1”
–  Transaction Logging & Roll-back operation
–  Session Termination
–  How many bytes are used?
»  1, 2, 4, less than 1 byte (6 bits), ….
–  How about byte order?
»  Little Endian / Big Endian
–  How about pre-transmission bit order?
! Communication conceptual definition and
fundamental processing from the application
viewpoint
Information Network1
Information Network1
»  MSB first, LSB first
5
Information Network1
6
! Framework protocol for concrete applications
–  note that an application protocol is not an application
–  SMTP (simple mail transfer protocol) is defined for email
exchange
–  However, there exist many email applications
•  MTA: sendmail, qmail, postfix, etc….
•  MUA: Eudora, Mozilla Thunderbird, MS/Outlook, etc….
Information Network1
7
Information Network1
8
! Communication Model
–  Client / Server model
–  Broadcast based parallelization
–  Peer-to-Peer (P2P) model
server
client
Request/reply
! Process Model
–  Side effect / no side effect
•  Process semantics
–  Atomic / checkpoint
client
! Functions
–  End user services
–  Network management
Information Network1
9
! Easy understanding of functional separation
Information Network1
10
! The server bears all of the processing bottlenecks
–  User interface: client
–  Each process: server
–  needs a server scale expansion to build large-scale services
–  guarantees good scalability
–  many various challenges for large-scale services
•  function aggregation, centralized resource management
•  manageable structure
! Does not match the bidirectional information
exchange model
! Easy service discovery model
–  Few servers responding to many clients
–  Service discovered by Well-known address （server） + wellknown port (service type）
–  Dynamic binding was difficult
–  the client makes use of information located at the server
–  active use of client-side resources and services such as
client-side injections of large quantities of information is
unreasonable
•  naming model and management model were not decided
•  scalability maintenance is difficult
•  currently, dynamic binding is widely used (Sun RPC bind etc.)
Information Network1
11
Information Network1
12
! Broadcast type
! Commands are sent to “up & running” hosts using
UDP broadcast
–  periodical information exchange through broadcast
–  sharing information between all processes
–  can discover active hosts
–  process on top of host affiliated to same port performs same
processing ! broadcast for parallel processing
–  using hosts that exist in same segment
–  e.g） rwhod
! Resolve limitations of being in the same segment by
using TCP
–  server federation originated in this technology space
Information Network1
13
Information Network1
14
! Direct information exchange of user-owned resources
–  all systems are both server and client
server
client
! Extremely popular
–  Gnutella, Winny, …..
server
! Issues : information discovery, security assurance of
the whole system
Request/reply
server
Information Network1
15
! Widespread perception by P2P service expansion
! Network structure built on top of another real network
at the application layer
–  virtualization of network structure
Information Network1
16
! Communication fails
–  Packet loss
–  Packet duplication
–  Complex processing due to retransmission
! Side effects of processing
IP layer
Information Network1
17
–  How to keep idempotence of server response?
–  Correct understanding about identification
–  Purpose of execution
•  Exactly once semantics
•  At least once semantics
•  At most once semantics
Information Network1
18
! “idempotent”
–  client always gets the same results, when the same requests
are processed by the server
–  we can build a simple retransmission system by creating
such processing system
•  pay attention to processing that alter states
! Parallelization of idempotent processing is easy
! Enables large-scale processing by decomposing the
server inner structure
look like one server The Internet
Idempotent processing Side-effect processing
•  load balancer
•  cluster
•  valid for load balance and fault
management
Information Network1
19
•  process aggregation
•  target : DB etc
Information Network1
20
! Non structured
–  Binary
–  ASCII
! Structured
–  TLV: Type, Length, Value
–  ASN.1
–  XML
Information Network1
21
ease of treatment / power of expression
Information Network1
!  Binary
!  ASCII
–  IP header (RFC791), TCP
header (RFC793)
–  defines difference between
host bit order and network
bit order
–  adjustment of integer value
manipulation
XML
ASN.1
ASCII
22
–  FTP (RFC959)
–  SMTP (RFC2821)
–  POP3 (RFC1939) …
–  ensure data transmission
even for channel that is not
8-bit clean
TLV
binary
bandwidth efficiency
Information Network1
23
Information Network1
24
001.204.01754: 220 ftp.isi.edu NcFTPd Server (free educational license) ready.
176.020.00021: USER anonymous
001.204.01754: 331 Guest login ok, send your complete e-mail address as password.
176.020.00021: PASS -wget@
001.204.01754: 230 Logged in anonymously.
176.020.00021: SYST
001.204.01754: 215 UNIX Type: L8
176.020.00021: PWD
001.204.01754: 257 "/" is cwd.
176.020.00021: TYPE I
001.204.01754: 200 Type okay.
176.020.00021: CWD /in-notes
001.204.01754: 250 "/in-notes" is new cwd.
Information Network1
1758: 220 ns.ixj-mc.com ESMTP Sendmail 8.9.3p2+3.1W/3.7W/ns; Fri, 30 May
2003 00:15:43 +09
0025: EHLO mf.aist-nara.ac.jp
1758: 250 ns.ixj-mc.com Hello 168.pool3.ftthtokyo.att.ne.jp [165.76.67.168], pleased
to meet you
0025: MAIL FROM:<[email protected]> SIZE=1524
1758: 250 <[email protected]>... Sender ok
0025: RCPT TO:<[email protected]>
1758: 250 <[email protected]>... Recipient ok
0025: DATA
1758: 354 Enter mail, end with "." on a line by itself
0025: Received: from mf.aist-nara.ac.jp (localhost [127.0.0.1])
3 +0900 (JST)
25
! Type, Length, Value
Information Network1
! OSPF (RFC2328)
! RADIUS (RFC2138)
! XDR (eXtended Data Representation) for Sun RPC/
NFS (1980’s)
! ASN.1 (1980’s)
! XML (1990’s)
! Structured information expression
! Requirements
26
–  wire efficiency
–  ease of treatment
–  description
–  Type-dependent processing
–  Enables to manipulate flexible-length data by inputting
Length information
–  Value defines each expression format
•  static typing, name space
•  data self-description
–  enhancement of tools and libraries
Information Network1
27
! (tag, length, value)
Information Network1
28
! Iso(1).org(3).dod(6).internet(1)
–  Tag: ASN.1 type
–  Length: size of the ASN.1 value
–  Value: ASN.1 value
–  Mgmt (2)
–  Experimental (3)
–  Private (4)
! 1.3.6.1.2.1….
–  defines data structure used in SNMP
–  MIB-II (RFC1214),
–  Structure of Management Information version 2 (SMIv2)
(RFC2578)
! ASN.1 type
– 
– 
– 
– 
INTEGER
OCTET STRING
OBJECT IDENTIFIER
SEQUENCE (array)
Information Network1
29
Information Network1
30
<?xml version='1.0' encoding='UTF-8'?>
<soap:Envelope xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/'
xmlns:xsi='http://www.w3.org/1999/XMLSchema-instance'
xmlns:xsd='http://www.w3.org/1999/XMLSchema'
xmlns:soapenc='http://schemas.xmlsoap.org/soap/encoding/'
soap:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'>
<soap:Body>
<n:getQuoteResponse xmlns:n='urn:xmethods-delayed-quotes'>
<Result xsi:type='xsd:float'>7.92</Result>
! <?xml version = "1.0"?>
! <tag attribute="value">
! <another-tag another-attribute="value" />
! </tag attribute="value">
</n:getQuoteResponse>
</soap:Body>
</soap:Envelope>
Information Network1
31
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsd="http://www.w3.org/1999/XMLSchema“
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<namesp1:getQuote xmlns:namesp1="urn:xmethods-delayed-quotes">
Information Network1
32
! Data self-description: RDF
! Enhancement of tools : XPath, XSLT,
! Enhancement of APIs: DOM, SAX, etc.
! Signature, encryption
–  XML Digital Signature
–  XML Encryption
<symbol xsi:type="xsd:string">AKAM</symbol>
</namesp1:getQuote>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Information Network1
33
Information Network1
34
35
Information Network1
36
! Systems using XML are widely used
–  expression in XML
–  XML processing system
! XML is common knowledge for network-related
service creation
–  XML is a required item
–  XML has been widely used for data expression in other
software
–  soon expands to a standard method
–  Programming Language＋XML
Information Network1
client / server
!  Warez exchange and sharing
Hybrid P2P
Pure P2P
–  warez file exchange and sharing by ftp, www and netnews
–  does not come under the spotlight due to its small scale
center
: Server
!  Appearance of file exchange application
center
: Server
–  In the early days
•  appearance of P2P application using Gnutella and Napster
–  Middle and late period
peer
•  towards a refined shape as WinMX () and KaZaa ()
–  hybrid P2P
centralized rendezvous
and communication
with the server
!  Shift to file sharing
–  appearance of Winny (2002)
–  towards pure P2P
–  more effective distribution by broadband
Information Network1
FTP, WWW
37
peer
peer
rendezvous at server,
communication
between peers
both rendezvous and
communication
between peers
WinMX, KaZaa
Gnutella, Freenet,
Winny
Information Network1
38
!  Features
! Napster
! Gnutella
! Winny
! WinMX
! LimeWire
! Share
! BitTorrent
! Cabos
–  sharing mp3 file lists between peers and enabling exchange of
these files
–  Hybrid P2P
–  chat application was implemented to Napster as a way file
exchange purpose communication tool
!  Biography
–  Jun, 1999 developed by Shawn Fanning (Northeastern University)
–  Jul, 2000 sued in court by RIAA , Napster network was stopped
–  Oct, 2003 restored as a music distribution service
Information Network1
39
Information Network1
40
!  What is Winny
! Features
–  enables publication of not only mp3 files but also various file
formats
–  no management server and no management organization
–  implementation of pure P2P
–  became the reference model of P2P file sharing software
–  file sharing application successor of WinMX was developed by a
Japanese individual
–  file sharing network software using pure P2P
–  Winny became extremely popular due to its ease of use, and a file
sharing network was built
!  Biography
! Biography
–  6, May, 2002 " version is released
–  5, May, 2003 2.0" version is released
–  28, Nov, 2003 arrest of a Winny user, developer’s house-search,
development is interrupted
–  31, May, 2004 developer is indicted by the Kyoto district public prosecutors
office, on grounds of a Copyright Act infringement
–  1, Jun, 2004 bailed out
–  13, Dec, 2006 court-ordered at Kyoto district court, found guilty (fined),
appealed the same day
–  Feb, 2000 released by developers of Nullsoft
–  Mar, 2000 released, but stopped release by AOL for 24
hours
–  various clones exists
Information Network1
41
Information Network1
42
! Issues on the quality of the exchanged information
! 2003 estimated 200,000 (network company survey)
– 2 millions (ACCS survey) users
! Nov, 2004 domestic traffic was reduced by 1/6
! July, 2006 estimated 1.75 millions user
–  distribution of copyrighted works
–  distribution of counterfeited files and faked files
–  spread of viruses
! Arising from file management
–  due to the use of file as the information unit
• 
• 
• 
• 
anyone can freely share a file
can not delete a shared file
impossible to identify the sharing source of a file
has no way of knowing the reliability of a targeted file
–  needs to exchange more structured information
Information Network1
43
Information Network1
44
! Expectations for P2P Distribution Network
! Potential customers
– 
– 
– 
– 
–  has proved existence of a consumer layer that can be
contents purchasers given appropriate price and availability
! Construction of an autonomous distribution network
by users
–  showed new solution for maintenance of distribution network
Information Network1
45
! Aggressive use of P2P network
File preservation
User preservation
Accounting
Traceability of file
Information Network1
46
! Segregation between physical network structure and
application layer structure
–  Oct, 2005 Nasper Japan is established by Tower Record
and Nasper
–  Kazza create business model by the distribution of
promotion files, the delivery of pay files and advertisement
–  appearance of usable bandwidth stress state
! Is it possible to build overlay-friendly underlying
networks?
! Skype
–  voice communication service by Gnutella technologies
–  is widely used all over the world
Information Network1
! Requirements for P2P distribution network
– 
– 
– 
– 
! possible information distribution infrastructure by
introducing appropriate technologies
cost reduction due to flow disintermediation
direct distribution from creator to user
distribution maintenance of minor products
maintenance of distribution market
–  We can get a hint by elaborating overlay networks
–  Research is also progressing on what kind of hinds can be
obtained
47
FY2010 情報ネットワーク論I 第12回
48
! E.g., in the case of an information storage service built
as an overlay network. Which node hold the desired
information?
! There exist many issues
–  Potential search halt
•  Does the search has been completed within the configured time?
•  Is there any efficient information search method?
–  Existence of hindering nodes
•  What to do if some nodes do not relay the search?
•  It is assumed that each node is not under the provider control
–  What to do when search results include down nodes?
FY2010 情報ネットワーク論I 第12回
49
! a few servers for many clients
Information Network1
50
! Required mechanism for load balancing
! Load balancer is widely used
! Operating principle
the whole is visible as a single server
–  LB manages termination point of TCP connection
The Internet
Server1: 192.168.20.1
Client: 163.221.50.15
Virtual server: 202.247.15.1
The Internet
Idempotent processing
Side-effect processing
•  load balancer
•  cluster
•  valid for load balance and fault
management
•  process aggregation
•  target : DB etc
Information Network1
51
! Which servers ended the connection?
Forwarding Table
Src: 163.221.50.15
Dst: 192.168.20.[1-3]
Server2: 192.168.20.2
Server3: 192.168.20.3
Information Network1
52
! Server does not need to exist in the same network
–  Various methods were designed and implemented
–  show multiple servers as a single one by using tunneling
–  sometimes this technology is called “server federation”
•  round robin, minimum connection, performance observation
–  fault management is also important
! CDN (Contents Delivery Network)
•  how to detects when a server is down?
–  active load balancing has been generalized in ISPs
–  Akamai.com is the pioneer of business service about CDN.
! How to hide the difference of server addresses
–  When server-side uses a private IP address, we must rewrite
address in communication
–  No simple NAT
•  sometimes address information is embedded into Cookie or
HTTP payload
•  some application protocols has side effects
Information Network1
53
Information Network1
54
new contents are
provided by server at
data center(freshness
control)
server
Data Center
IX
clients
individual ISPs provides
contents for customer by CD
server
(well-managed access)
clients
Information Network1
55
Information Network1
! load balance by routing information
! Root DNS server is a typical example
56
2001::1
HOST
HOST
HOST
HOST
2001::1
2001::1
Information Network1
57
Information Network1
58
FY2010 情報ネットワーク論I 第12回
60
!  F root DNS
–  Originally at ISC in Silicon
Valley, Calif.
–  Currently, Hong Kong &
Amsterdam
?
? ?
client request reaches the
neighbor server that is on
the shortest path depending
on network routing
Hong Kong, CN
each server, while using the same
address, distributes route
information from each location.
Amsterdam, NL
Silicon Valley, Calif.
Information Network1
59
! Real-Time Media Transmission Technology for voice
and video transmission
! Internet: to support the real economy
–  The Internet is a social infrastructure for trading money
–  The security technologies become essential for the
protection of assets
–  Real-Time Packet Transmission over Network Layer and
Transport Layer
–  Coding Techniques which are adapted to media
characteristics at the upper layer
–  Providing and sharing SDK (Software Development Kit) for
building real applications
FY2010 情報ネットワーク論I 第12回
! Encryption, Authentication Technology, ID
Management Technology
! Secure Distributed Information Management
! Service Backup for BCM (Business Continuity
Management)
! Standardization of application protocols implementing
the above services has started.
61
FY2010 情報ネットワーク論I 第12回
62