Solaris Internals
Solaris 10 and OpenSolaris Kernel Architecture (paperback)
2nd Edition
Published on 5. June 2015
Book
Paperback/Softback
1072 pages
978-0-13-418597-2 (ISBN)
Description
"The Solaris (TM)Internals volumes are simply the best and most comprehensive treatment of the Solaris (and OpenSolaris) Operating Environment. Any person using Solaris--in any capacity--would be remiss not to include these two new volumes in their personal library. With advanced observability tools in Solaris (likeDTrace), you will more often find yourself in what was previously unchartable territory. Solaris (TM) Internals, Second Edition, provides us a fantastic means to be able to quickly understand these systems and further explore the Solaris architecture--especially when coupled with OpenSolaris source availability."
--Jarod Jenson, chief systems architect, Aeysis
"The Solaris (TM) Internals volumes by Jim Mauro and Richard McDougall must be on your bookshelf if you are interested in in-depth knowledge of Solaris operating system internals and architecture. As a senior Unix engineer for many years, I found the first edition of Solaris (TM) Internals the only fully comprehensive source for kernel developers, systems programmers, and systems administrators. The new second edition, with the companion performance and debugging book, is an indispensable reference set, containing many useful and practical explanations of Solaris and its underlying subsystems, including tools and methods for observing and analyzing any system running Solaris 10 or OpenSolaris."
--Marc Strahl, senior UNIX engineer
Solaris (TM) Internals, Second Edition, describes the algorithms and data structures of all the major subsystems in the Solaris 10 and OpenSolaris kernels. The text has been extensively revised since the first edition, with more than 600 pages of new material. Integrated Solaris tools and utilities, including DTrace, MDB, kstat, and the process tools, are used throughout to illustrate how the reader can observe the Solaris kernel in action. The companion volume, Solaris (TM) Performance and Tools, extends the examples contained here, and expands the scope to performance and behavior analysis. Coverage includes:
Virtual and physical memory
Processes, threads, and scheduling
File system framework and UFS implementation
Networking: TCP/IP implementation
Resource management facilities and zones
The Solaris (TM) Internals volumes make a superb reference for anyone using Solaris 10 and OpenSolaris.
--Jarod Jenson, chief systems architect, Aeysis
"The Solaris (TM) Internals volumes by Jim Mauro and Richard McDougall must be on your bookshelf if you are interested in in-depth knowledge of Solaris operating system internals and architecture. As a senior Unix engineer for many years, I found the first edition of Solaris (TM) Internals the only fully comprehensive source for kernel developers, systems programmers, and systems administrators. The new second edition, with the companion performance and debugging book, is an indispensable reference set, containing many useful and practical explanations of Solaris and its underlying subsystems, including tools and methods for observing and analyzing any system running Solaris 10 or OpenSolaris."
--Marc Strahl, senior UNIX engineer
Solaris (TM) Internals, Second Edition, describes the algorithms and data structures of all the major subsystems in the Solaris 10 and OpenSolaris kernels. The text has been extensively revised since the first edition, with more than 600 pages of new material. Integrated Solaris tools and utilities, including DTrace, MDB, kstat, and the process tools, are used throughout to illustrate how the reader can observe the Solaris kernel in action. The companion volume, Solaris (TM) Performance and Tools, extends the examples contained here, and expands the scope to performance and behavior analysis. Coverage includes:
Virtual and physical memory
Processes, threads, and scheduling
File system framework and UFS implementation
Networking: TCP/IP implementation
Resource management facilities and zones
The Solaris (TM) Internals volumes make a superb reference for anyone using Solaris 10 and OpenSolaris.
Reviews / Votes
In total, the two books Solaris Performance and Tools & Solaris Internals reviewed here present a new level of knowledge about the internals of Solaris, what they do, how they behave, and how to analyze that behavior. The books are a must for developers, system programmers, and systems administrators who work with Solaris 8, 9, or 10. They are especially useful for users of Solaris 10 and OpenSolaris because of their exploration of the new tools in those releases. These books receive my highest recommendation.More details
Other editions
Previous edition
Jim Mauro | Richard Mcdougall | Sun Microsystems Press
Solaris (TM) Internals
Book
11/2000
Prentice Hall
€59.41
Article exhausted; check for reprint
Persons
Richard McDougall is a Distinguished Engineer at Sun Microsystems, specializing in operating systems technology and systems performance.
Jim Mauro is a Senior Staff Engineer in the Performance, Architecture, and Applications Engineering group at Sun Microsystems, where his most recent efforts have focused on Solaris performance on Opteron platforms.
Jim Mauro is a Senior Staff Engineer in the Performance, Architecture, and Applications Engineering group at Sun Microsystems, where his most recent efforts have focused on Solaris performance on Opteron platforms.
Content
Foreword xxvii Preface xxix About the Authors xxxvii Acknowledgments xxxix Part One: Introduction to Solaris Internals 1 Chapter 1: Introduction 3 1.1 Key Features of Solaris 10, Solaris 9, and Solaris 8 4
1.2 Key Differentiators 12
1.3 Kernel Overview 15
1.4 Processes, Threads, and Scheduling 18
1.5 Interprocess Communication 23
1.6 Signals 25
1.7 Memory Management 26
1.8 Files and File Systems 29
1.9 Resource Management 30
Part Two: The Process Model 41 Chapter 2: The Solaris Process Model 43 2.1 Components of a Process 44
2.2 Process Model Evolution 48
2.3 Executable Objects 52
2.4 Process Structures 55
2.5 Kernel Process Table 79
2.6 Process Resource Attributes 84
2.7 Process Creation 89
2.8 System Calls 98
2.9 Process Termination 106
2.10 The Process File System 110
2.11 Signals 129
2.12 Sessions and Process Groups 150
2.13 MDB Reference 156
Chapter 3: Scheduling Classes and the Dispatcher 157 3.1 Fundamentals 157
3.2 Processor Abstractions 162
3.3 Dispatcher Queues, Structures, and Variables 171
3.4 Dispatcher Locks 183
3.5 Dispatcher Initialization 190
3.6 Scheduling Classes 192
3.7 Thread Priorities 207
3.8 Dispatcher Functions 234
3.9 Preemption 246
3.10 The Kernel Sleep/Wakeup Facility 253
3.11 Interrupts 262
3.12 Summary 270
3.13 MDB Reference 271
Chapter 4: Interprocess Communication 273 4.1 The System V IPC Framework 274
4.2 System V IPC Resource Controls 282
4.3 Configuring IPC Tuneables on Solaris 10 285
4.4 System V Shared Memory 286
4.5 System V Semaphores 295
4.6 System V Message Queues 299
4.7 POSIX IPC 303
4.8 Solaris Doors 312
4.9 MDB Reference 321
Chapter 5: Process Rights Management 323 5.1 Then and Now 323
5.2 Least Privilege in Solaris 324
5.3 Process Privilege Models 325
5.4 Privilege Awareness: The Details 334
5.5 Least Privilege Interfaces 344
Part Three: Resource Management 365 Chapter 6: Zones 367 6.1 Introduction 367
6.2 Zone Runtime 371
6.3 Booting Zones 375
6.4 Security 379
6.5 Process Model 386
6.6 File Systems 389
6.7 Networking 393
6.8 Devices 398
6.9 Interprocess Communication 405
6.10 Resource Management and Observability 407
6.11 MDB Reference 414
Chapter 7: Projects, Tasks, and Resource Controls 415 7.1 Projects and Tasks Framework 415
7.2 The Project Database 418
7.3 Project and Task APIs 419
7.4 Kernel Infrastructure for Projects and Tasks 420
7.5 Resource Controls 423
7.6 Interfaces for Resource Controls 432
7.7 Kernel Interfaces for Resource Controls 437
Part Four: Memory 445 Chapter 8: Introduction to Solaris Memory 447 8.1 Virtual Memory Primer 447
8.2 Two Levels of Memory 448
8.3 Memory Sharing and Protection 448
8.4 Pages: Basic Units of Physical Memory 448
8.5 Virtual-to-Physical Translation 449
8.6 Physical Memory Management: Paging and Swapping 450
8.7 Virtual Memory as a File System Cache 450
8.8 New Features of the Virtual Memory Implementation 451
Chapter 9: Virtual Memory 455 9.1 Design Overview 455
9.2 Virtual Address Spaces 457
9.3 Tracing the VM System 466
9.4 Virtual Address Space Management 467
9.5 Segment Drivers 476
9.6 Anonymous Memory 485
9.7 The Anonymous Memory Layer 487
9.8 The swapfs Layer 489
9.9 Virtual Memory Watchpoints 492
9.10 Changes to Support Large Pages 494
9.11 MDB Reference 501
Chapter 10: Physical Memory 503 10.1 Physical Memory Allocation 503
10.2 Pages: The Basic Unit of Solaris Memory 506
10.3 The Page Scanner 516
10.4 MDB Reference 525
Chapter 11: Kernel Memory 527 11.1 Kernel Virtual Memory Layout 527
11.2 Kernel Memory Allocation 534
11.3 The Vmem Allocator 552
11.4 Kernel Memory Allocator Tracing 562
11.5 MDB Reference 578
Chapter 12: Hardware Address Translation 581 12.1 HAT Overview 581
12.2 The UltraSPARC HAT Layer 583
12.3 The x64 HAT Layer 625
12.4 MDB Reference 636
Chapter 13: Working with Multiple Page Sizes in Solaris 639 13.1 Determining When to Use Large Pages 639
13.2 Measuring Application Performance 640
13.3 Configuring for Multiple Page Sizes 645
Part Five: File Systems 655 Chapter 14: File System Framework 657 14.1 File System Framework 657
14.2 Process-Level File Abstractions 658
14.3 Solaris File System Framework 668
14.4 File System Modules 672
14.5 The Virtual File System (vfs) Interface 675
14.6 The Vnode 685
14.7 File System I/O 707
14.8 File Systems and Memory Allocation 718
14.9 Path-Name Management 722
14.10 The Directory Name Lookup Cache 726
14.11 The File System Flush Daemon 734
14.12 File System Conversion to Solaris 10 734
14.13 MDB Reference 736
Chapter 15: The UFS File System 737 15.1 UFS Development History 737
15.2 UFS On-Disk Format 739
15.3 The UFS Inode 751
15.4 Access Control in UFS 764
15.5 Extended Attributes in UFS 767
15.6 Locking in UFS 768
15.7 Logging 775
15.8 MDB Reference 790
Part Six: Platform Specifics 793 Chapter 16: Support for NUMA and CMT Hardware 795 16.1 Memory Hierarchy Designs 796
16.2 Memory Placement Optimization Framework 799
16.3 Initial Thread Placement 802
16.4 Scheduling 802
16.5 Memory Allocation 803
16.6 Lgroup Implementation 804
16.7 MPO APIs 807
16.8 Locality Group Hierarchy 811
16.9 MPO Statistics 813
16.10 MDB Reference 814
Chapter 17: Locking and Synchronization 815 17.1 Synchronization 815
17.2 Parallel Systems Architectures 816
17.3 Hardware Considerations for Locks and Synchronization 819
17.4 Introduction to Synchronization Objects 824
17.5 Mutex Locks 827
17.6 Reader/Writer Locks 835
17.7 Turnstiles and Priority Inheritance 840
17.8 Kernel Semaphores 844
17.9 DTrace Lockstat Provider 846
Part Seven: Networking 853 Chapter 18: The Solaris Network Stack 855 18.1 STREAMS and the Network Stack 855
18.2 Solaris 10 Stack: Design Goals 862
18.3 Solaris 10 Network Stack Framework 863
18.4 TCP as an Implementation of the New Framework 870
18.5 UDP 875
18.6 Synchronous STREAMS 878
18.7 IP 880
18.8 Solaris Device Driver Framework 882
18.9 Interrupt Model and NIC Speeds 891
18.10 Summary 895
18.11 MDB Reference 895
Part Eight: Kernel Services 899 Chapter 19: Clocks and Timers 901 19.1 The System Clock Thread 901
19.2 Callouts and Callout Tables 904
19.3 System Time Facilities 910
19.4 The Cyclic Subsystem 912
Chapter 20: Task Queues 927 20.1 Overview of Task Queues 927
20.2 Dynamic Task Queues 928
20.3 Task Queues Kernel Programming Interfaces 932
20.4 Device Driver Interface for Task Queues 934
20.5 Task Queue Observability 935
20.6 Task Queue Implementation Notes 937
Chapter 21: kmdb Implementation 943 21.1 Introduction 943
Appendices 963 Appendix A: Kernel Virtual Address Maps 965 Appendix B: Adding a System Call to Solaris 971 Appendix C: A Sample Procfs Utility 975 Bibliography 979 Index 983
1.2 Key Differentiators 12
1.3 Kernel Overview 15
1.4 Processes, Threads, and Scheduling 18
1.5 Interprocess Communication 23
1.6 Signals 25
1.7 Memory Management 26
1.8 Files and File Systems 29
1.9 Resource Management 30
Part Two: The Process Model 41 Chapter 2: The Solaris Process Model 43 2.1 Components of a Process 44
2.2 Process Model Evolution 48
2.3 Executable Objects 52
2.4 Process Structures 55
2.5 Kernel Process Table 79
2.6 Process Resource Attributes 84
2.7 Process Creation 89
2.8 System Calls 98
2.9 Process Termination 106
2.10 The Process File System 110
2.11 Signals 129
2.12 Sessions and Process Groups 150
2.13 MDB Reference 156
Chapter 3: Scheduling Classes and the Dispatcher 157 3.1 Fundamentals 157
3.2 Processor Abstractions 162
3.3 Dispatcher Queues, Structures, and Variables 171
3.4 Dispatcher Locks 183
3.5 Dispatcher Initialization 190
3.6 Scheduling Classes 192
3.7 Thread Priorities 207
3.8 Dispatcher Functions 234
3.9 Preemption 246
3.10 The Kernel Sleep/Wakeup Facility 253
3.11 Interrupts 262
3.12 Summary 270
3.13 MDB Reference 271
Chapter 4: Interprocess Communication 273 4.1 The System V IPC Framework 274
4.2 System V IPC Resource Controls 282
4.3 Configuring IPC Tuneables on Solaris 10 285
4.4 System V Shared Memory 286
4.5 System V Semaphores 295
4.6 System V Message Queues 299
4.7 POSIX IPC 303
4.8 Solaris Doors 312
4.9 MDB Reference 321
Chapter 5: Process Rights Management 323 5.1 Then and Now 323
5.2 Least Privilege in Solaris 324
5.3 Process Privilege Models 325
5.4 Privilege Awareness: The Details 334
5.5 Least Privilege Interfaces 344
Part Three: Resource Management 365 Chapter 6: Zones 367 6.1 Introduction 367
6.2 Zone Runtime 371
6.3 Booting Zones 375
6.4 Security 379
6.5 Process Model 386
6.6 File Systems 389
6.7 Networking 393
6.8 Devices 398
6.9 Interprocess Communication 405
6.10 Resource Management and Observability 407
6.11 MDB Reference 414
Chapter 7: Projects, Tasks, and Resource Controls 415 7.1 Projects and Tasks Framework 415
7.2 The Project Database 418
7.3 Project and Task APIs 419
7.4 Kernel Infrastructure for Projects and Tasks 420
7.5 Resource Controls 423
7.6 Interfaces for Resource Controls 432
7.7 Kernel Interfaces for Resource Controls 437
Part Four: Memory 445 Chapter 8: Introduction to Solaris Memory 447 8.1 Virtual Memory Primer 447
8.2 Two Levels of Memory 448
8.3 Memory Sharing and Protection 448
8.4 Pages: Basic Units of Physical Memory 448
8.5 Virtual-to-Physical Translation 449
8.6 Physical Memory Management: Paging and Swapping 450
8.7 Virtual Memory as a File System Cache 450
8.8 New Features of the Virtual Memory Implementation 451
Chapter 9: Virtual Memory 455 9.1 Design Overview 455
9.2 Virtual Address Spaces 457
9.3 Tracing the VM System 466
9.4 Virtual Address Space Management 467
9.5 Segment Drivers 476
9.6 Anonymous Memory 485
9.7 The Anonymous Memory Layer 487
9.8 The swapfs Layer 489
9.9 Virtual Memory Watchpoints 492
9.10 Changes to Support Large Pages 494
9.11 MDB Reference 501
Chapter 10: Physical Memory 503 10.1 Physical Memory Allocation 503
10.2 Pages: The Basic Unit of Solaris Memory 506
10.3 The Page Scanner 516
10.4 MDB Reference 525
Chapter 11: Kernel Memory 527 11.1 Kernel Virtual Memory Layout 527
11.2 Kernel Memory Allocation 534
11.3 The Vmem Allocator 552
11.4 Kernel Memory Allocator Tracing 562
11.5 MDB Reference 578
Chapter 12: Hardware Address Translation 581 12.1 HAT Overview 581
12.2 The UltraSPARC HAT Layer 583
12.3 The x64 HAT Layer 625
12.4 MDB Reference 636
Chapter 13: Working with Multiple Page Sizes in Solaris 639 13.1 Determining When to Use Large Pages 639
13.2 Measuring Application Performance 640
13.3 Configuring for Multiple Page Sizes 645
Part Five: File Systems 655 Chapter 14: File System Framework 657 14.1 File System Framework 657
14.2 Process-Level File Abstractions 658
14.3 Solaris File System Framework 668
14.4 File System Modules 672
14.5 The Virtual File System (vfs) Interface 675
14.6 The Vnode 685
14.7 File System I/O 707
14.8 File Systems and Memory Allocation 718
14.9 Path-Name Management 722
14.10 The Directory Name Lookup Cache 726
14.11 The File System Flush Daemon 734
14.12 File System Conversion to Solaris 10 734
14.13 MDB Reference 736
Chapter 15: The UFS File System 737 15.1 UFS Development History 737
15.2 UFS On-Disk Format 739
15.3 The UFS Inode 751
15.4 Access Control in UFS 764
15.5 Extended Attributes in UFS 767
15.6 Locking in UFS 768
15.7 Logging 775
15.8 MDB Reference 790
Part Six: Platform Specifics 793 Chapter 16: Support for NUMA and CMT Hardware 795 16.1 Memory Hierarchy Designs 796
16.2 Memory Placement Optimization Framework 799
16.3 Initial Thread Placement 802
16.4 Scheduling 802
16.5 Memory Allocation 803
16.6 Lgroup Implementation 804
16.7 MPO APIs 807
16.8 Locality Group Hierarchy 811
16.9 MPO Statistics 813
16.10 MDB Reference 814
Chapter 17: Locking and Synchronization 815 17.1 Synchronization 815
17.2 Parallel Systems Architectures 816
17.3 Hardware Considerations for Locks and Synchronization 819
17.4 Introduction to Synchronization Objects 824
17.5 Mutex Locks 827
17.6 Reader/Writer Locks 835
17.7 Turnstiles and Priority Inheritance 840
17.8 Kernel Semaphores 844
17.9 DTrace Lockstat Provider 846
Part Seven: Networking 853 Chapter 18: The Solaris Network Stack 855 18.1 STREAMS and the Network Stack 855
18.2 Solaris 10 Stack: Design Goals 862
18.3 Solaris 10 Network Stack Framework 863
18.4 TCP as an Implementation of the New Framework 870
18.5 UDP 875
18.6 Synchronous STREAMS 878
18.7 IP 880
18.8 Solaris Device Driver Framework 882
18.9 Interrupt Model and NIC Speeds 891
18.10 Summary 895
18.11 MDB Reference 895
Part Eight: Kernel Services 899 Chapter 19: Clocks and Timers 901 19.1 The System Clock Thread 901
19.2 Callouts and Callout Tables 904
19.3 System Time Facilities 910
19.4 The Cyclic Subsystem 912
Chapter 20: Task Queues 927 20.1 Overview of Task Queues 927
20.2 Dynamic Task Queues 928
20.3 Task Queues Kernel Programming Interfaces 932
20.4 Device Driver Interface for Task Queues 934
20.5 Task Queue Observability 935
20.6 Task Queue Implementation Notes 937
Chapter 21: kmdb Implementation 943 21.1 Introduction 943
Appendices 963 Appendix A: Kernel Virtual Address Maps 965 Appendix B: Adding a System Call to Solaris 971 Appendix C: A Sample Procfs Utility 975 Bibliography 979 Index 983