A Global Storage
System for User
Generated Contents
Prof. Xiaohua Jia, City University of Hong Kong
Dr. Jianfei He, Huawei Technologies Inc.
Dr. Bo Zhang, City University of Hong Kong
Outline
o Motivations and Objectives
o System Architecture
o System Design and Implementation
o Conclusion and Demo
Motivation
Application service providers (ASPs) become the de facto
owners of users’ data and social relations because of
hosting and managing them.
Facebook Server
(Frontend)
Facebook Storage
(UGC / relations)
Sina Weibo Server
(Frontend)
Sina Weibo Storage
(UGC / relations)
User
Motivation
What are the consequences?
o Big ASPs control users’ data and social relations and hijack the
users to go with them
o Medium and small ASPs are denied of opportunities to enter
the business
o User’s social relations are fragmented on multiple ASPs
o Publishing/subscribing of user’s data is limited within the
scope of an ASP
o If the user publishes the data to multiple ASPs, it’s difficult to
withdraw the published data
System Background
o As the advent of NFV and ICNs, traditional network routers are
replaced by general purpose computer servers
o There are huge amount of under-utilized storage and computing
powers scattered all over the Internet
Design Objectives
o Decouple users' UGC from ASPs
The architecture is able to store the UGC, and deliver the content to
the targeted audiences.
o Decouple users’ social relations from ASPs
The information of users social relations is integrated into the
architecture for data publishing / subscribing.
o Enable both vertical and horizontal openness
 vertical openness: ASPs interact with the system
 horizontal openness: interoperate between autonomous domains
System Architecture
End Users
End Users
Processed Data
Presentation Layer: data processing and presentation
ASPs
APPs
Presentation Layer
ASPs or APPs: use data from the
information layer to provide
advanced services to end users.
Raw User Data
Information Layer: data service and networking service
...
VSS Domain 1
VSS Domain 2
VSS Domain N
Inter-domain interface (via Internet / direct connections)
Information Layer
Multi-domain VSS (Versatile
Storage System): responsible for
content storage, access control
publishing/subscribing, and
distribution.
Information Layer: decoupling of
user data and social relations
o Decouple UGC and users’ social relations from ASPs
- Provide storage space for users
- Manage social relations of users
- Provide fundamental content services to users: content storage,
distribution, retrieval, access control, publishing and subscribing, ……
Users’ social relations are decoupled from ASPs
Alice
Facebook Client App
1. Alice sends a “friend” request
to Bob in Facebook app
Weibo Client App
VSS architecture
3. Bob becomes Alice’s friend
in Facebook app
Bob
2. Bob sees Alice become his
follower in Weibo app, then
Bob follows Alice back.
Information Layer: decoupling of
user data and social relations
UGC is decoupled from ASPs
Alice
Facebook Client App
4. Alice posts a message to
“Friends” in Facebook app
Weibo Client App
VSS architecture
Bob
5. Bob can see the message
in Weibo app
Information Layer: open
interfaces
o Vertical openness
- Any ASPs can provide content services based on the information layer of VSS
- Provide standard APIs of VSS that is open to all ASPs
o Horizontal openness
- Composed of multiple autonomous service domains
- Interconnected by “thin” inter-domain interface
inter-domain interface
Domain 1 (China)
inter-domain interface
Domain 2 (UK)
Domain 3 (USA)
Information Layer: support of
network communications
o Data communication based on file systems
- Integrate data service with networking service
- Support general data communication
o Real-time communication
- Support inter-person real-time communication
Presentation Layer
o Content Presentation
- Present the contents in appropriate formats to end users
(e.g., content sorting, filtering, sorting, format conversion, etc.)
o Additional Features
- Provide functions that are not directly supported by Information Layer
(e.g., user / content user search & recommendation)
- Process advanced logics that are not handled by Information Layer
(e.g., comment handling)
Information
Layer
Presentation Layer: data processing and presentation
ASPs
APPs
End Users
System Designs
o
o
o
o
o
o
Fundamental services
Management of user data
Management of users social relations
“Thin” API interface
Roaming of users
Security and privacy
Fundamental Services of VSS
o VSS provides fundamental content and networking services to ASPs,
such as content management, users relations management, and
communications
o VSS decouples users’ data and social relations from ASPs
User Bob
User Alice
ASPs provide advanced /
customized services
Uses Facebook
Client App
Uses Weibo
Client App
VSS provides Fundamental operations
Application Services built on top
of VSS
o VSS can support a variety of services provided by
ASPs
- Instant message services: QQ, WeChat, Line,
WhatsApp
- Blog / microblog systems: Twitter, Weibo,
Blogger, Wordpress
- Social networks: Facebook, Qzone
- E-mail systems
- ……
Management of User Data
Users can use either a client application or a web interface (web
browser) to upload local files to or download files from the VSS system.
Client Application
User
VSS Domain
Web Interface
ASP Independent
publication / notification
o Alice (in China) shares a photo to her friends Bob and Cathy
o VSS China domain stores the photo and sends the notifications to the
home domains of Bob and Cathy respectively
o Bob & Cathy can see the photo via any application
o Once the photo is deleted by Alice, all references to this photo is
removed from the entire system
Bob @ USA
USA Domain
Alice @ China
China Domain
UK Domain
Cathy @ UK
Management of social
relations
VSS manages basic social relations among the users
o VSS manages the contact list, contact groups, and the mapping
between different contact groups and ASPs / APPs for users,
decoupling users’ social relations from ASPs
Contact-1
Stars
Twitter
Contact-2
Family
Sina Weibo
Contact-3
Classmates
Facebookk
Colleagues
Personal E-mail
Stranger
Work E-mail
Contact-n
Modeling of social relations
VSS aims to model general types of user social relations on
the Internet
o 1-way friendship: Weibo, Twitter, address book, contact list
o 2-way friendship: QQ, WeChat, Line, WhatsApp, Skype, Facebook
o Workgroups / teams: Dropbox shared folder, SkyDrive Groups, QQ
/ WeChat group chat
“Thin” interface: vertical /
horizontal
A “thin” interface shall be
o Simple – a minimum set of operations for both vertical interface
and horizontal interface
o Stable – remain stable for a relatively long period of time
o Complete – able to support a variety of applications
Information Layer
Domain 1
Domain 2
Data Storage
Fundamental Logics
Relationship Management
Real-time Communication
Data Storage
Inter-domain
Interface
via Internet /
direct connections
Fundamental Logics
Relationship Management
Real-time Communication
Handling of user roaming
The data accessed by a roaming user shall be transferred from its home
domain to the destination domain.
Alice @
Domain 1
Domain 3
Domain 1
Domain 2
Domain 4
Bob @
Domain 3
Roaming
Bob @
Domain 3
Domain 5
Handling of user roaming
Why consider user roaming?
o VSS consists of multiple autonomous domains
o A user’s information is stored and managed by his home domain
Requirements of roaming handling?
o Remote authentication of users
There shall be a simple and efficient method to authenticate a roaming user
so that the access permission can be assigned to the user
o Local & remote execution of commands
Some commands can only be executed locally (or passed back to the home
domain for execution if the user is in a remote domain) for security reasons
or performance reasons
o Caching and data pre-fetch
The domain shall be able to utilize its local cache to improve users’ roaming
experience
Security and privacy issues
Security issues
o Storage security: all stored data can be auto-encrypted (if users wish)
o Security for cross-domain interoperation and communications
Secure both the data storage and the communication channels!
Inter-domain channel
Security and privacy issues
Privacy concerns of cross-domain user operations
o protect privacy of communication parties
- Hide sender’s ID from the receiver’s domain
- Hide receiver’s ID from the sender’s domain
o Protect privacy of information consumers
- Hide subscriber’s ID from the foreign domain when subscribing a foreign
domains’ information
- Hide subscription details from subscriber’s home domain when
subscribing information from a foreign domain
o How?
- By using public / private key encryption
Conclusion
o VSS system decouples ASPs from users' data and users'
social relations
o With VSS system, ASPs have to rely on better quality
services to win user groups
o VSS integrates file services with traditional networking
services. It can be used as universal communication
platform
Demo
Demo settings:
o 3 end users
- Alice
- Bob
- Cathy
Alice
Bob
Cathy
FaceBlog
SinaBlog
NetDrive
o 3 ASPs / APPs
- FaceBlog
- SinaBlog
- NetDrive
o 2 domains
- Domain @ USA
- Domain @ China
VSS Domain (USA)
VSS Domain (China)
Thank You!