Security Intelligence

Foreword xv
Preface xvii
Chapter 1 Fundamentals of Secure Proxies 1
Security Must Protect and Empower Users 2
The Birth of Shadow IT 2
Internet of Things and Connected Consumer Appliances 3
Conventional Security Solutions 5
Traditional Firewalls: What Are Their Main Deficiencies? 5
Firewall with DPI: A Better Solution? 9
IDS/IPS and Firewall 11
Unified Threat Management and Next?]Generation Firewall 14
Security Proxy--A Necessary Extension of the End Point 15
Transaction?]Based Processing 18
The Proxy Architecture 19
SSL Proxy and Interception 22
Interception Strategies 24
Certificates and Keys 28
Certificate Pinning and OCSP Stapling 32
SSL Interception and Privacy 33
Summary 35
Chapter 2 Proxy Deployment Strategies and Challenges 37
Definitions of Proxy Types: Transparent Proxy and Explicit Proxy 38
Inline Deployment of Transparent Proxy: Physical Inline and Virtual Inline 41
Physical Inline Deployment 41
Virtual Inline Deployment 43
Traffic Redirection Methods: WCCP and PBR 44
LAN Port and WAN Port 46
Forward Proxy and Reverse Proxy 47
Challenges of Transparent Interception 48
Directionality of Connections 53
Maintaining Traffic Paths 53
Avoiding Interception 56
Asymmetric Traffic Flow Detection and Clustering 58
Proxy Chaining 62
Summary 64
Chapter 3 Proxy Policy Engine and Policy Enforcements 67
Policy System Overview 69
Conditions and Properties 70
Policy Transaction 71
Policy Ticket 73
Policy Updates and Versioning System 77
Security Implications 77
Policy System in the Cloud Security Operation 80
Policy Evaluation 82
Policy Checkpoint 82
Policy Execution Timing 84
Revisiting the Proxy Interception Steps 86
Enforcing External Policy Decisions 90
Summary 91
Chapter 4 Malware and Malware Delivery Networks 93
Cyber Warfare and Targeted Attacks 94
Espionage and Sabotage in Cyberspace 94
Industrial Espionage 96
Operation Aurora 96
Watering Hole Attack 98
Breaching the Trusted Third Party 100
Casting the Lures 101
Spear Phishing 102
Pharming 102
Cross?]Site Scripting 103
Search Engine Poisoning 106
Drive?]by Downloads and the Invisible iframe 109
Tangled Malvertising Networks 113
Malware Delivery Networks 114
Fast?]Flux Networks 117
Explosion of Domain Names 119
Abandoned Sites and Domain Names 120
Antivirus Software and End?]Point Solutions - The Losing Battle 121
Summary 122
Chapter 5 Malnet Detection Techniques 123
Automated URL Reputation System 124
Creating URL Training Sets 125
Extracting URL Feature Sets 126
Classifier Training 128
Dynamic Webpage Content Rating 131
Keyword Extraction for Category Construction 132
Keyword Categorization 135
Detecting Malicious Web Infrastructure 138
Detecting Exploit Servers through Content Analysis 138
Topology?]Based Detection of Dedicated Malicious Hosts 142
Detecting C2 Servers 144
Detection Based on Download Similarities 147
Crawlers 148
Detecting Malicious Servers with a Honeyclient 150
High Interaction versus Low Interaction 151
Capture?]HPC: A High?]Interaction Honeyclient 152
Thug: A Low?]Interaction Honeyclient 154
Evading Honeyclients 154
Summary 158
Chapter 6 Writing Policies 161
Overview of the ProxySG Policy Language 162
Scenarios and Policy Implementation 164
Web Access 164
Access Logging 167
User Authentication 170
Safe Content Retrieval 177
SSL Proxy 181
Reverse Proxy Deployment 183
DNS Proxy 187
Data Loss Prevention 188
E?]mail Filtering 190
A Primer on SMTP 191
E?]mail Filtering Techniques 200
Summary 202
Chapter 7 The Art of Application Classification 203
A Brief History of Classification Technology 204
Signature Based Pattern Matching Classification 206
Extracting Matching Terms - Aho?]Corasick Algorithm 208
Prefix?]Tree Signature Representation 211
Manual Creation of Application Signatures 214
Automatic Signature Generation 216
Flow Set Construction 218
Extraction of Common Terms 220
Signature Distiller 222
Considerations 225
Machine Learning?]Based Classification Technique 226
Feature Selection 228
Supervised Machine Learning Algorithms 232
Naive Bayes Method 233
Unsupervised Machine Learning Algorithms 236
Expectation?]Maximization 237
K?]Means Clustering 240
Classifier Performance Evaluation 243
Proxy versus Classifier 247
Summary 250
Chapter 8 Retrospective Analysis 251
Data Acquisition 252
Logs and Retrospective Analysis 253
Log Formats 254
Log Management and Analysis 255
Packet Captures 259
Capture Points 259
Capture Formats 261
Capture a Large Volume of Data 263
Data Indexing and Query 264
B?]tree Index 265
B?]tree Search 267
B?]tree Insertion 268
Range Search and B+?]tree 270
Bitmap Index 272
Bitmap Index Search 273
Bitmap Index Compression 276
Inverted File Index 279
Inverted File 279
Inverted File Index Query 281
Inverted File Compression 282
Performance of a Retrospective Analysis System 283
Index Sizes 283
Index Building Overhead 285
Query Response Delay 286
Scalability 288
Notes on Building a Retrospective Analysis System 289
MapReduce and Hadoop 289
MapReduce for Parallel Processing 292
Hadoop 293
Open Source Data Storage and Management Solution 295
Why a Traditional RDBMS Falls Short 295
NoSQL and Search Engines 296
NoSQL and Hadoop 297
Summary 298
Chapter 9 Mobile Security 299
Mobile Device Management, or Lack Thereof 300
Mobile Applications and Their Impact on Security 303
Security Threats and Hazards in Mobile Computing 304
Cross?]Origin Vulnerability 305
Near Field Communication 306
Application Signing Transparency 307
Library Integrity and SSL Verification Challenges 307
Ad Fraud 308
Research Results and Proposed Solutions 308
Infrastructure?]Centric Mobile Security Solution 311
Towards the Seamless Integration of WiFi and Cellular Networks 312
Security in the Network 313
Summary 315
Bibliography 317
Index 327