Page
1
Expert Scripting and Automation for SQL Server DBAs Amplifying Productivity Through Automation — Second Edition — Peter A. Carter
Page
2
Expert Scripting and Automation for SQL Server DBAs Amplifying Productivity Through Automation Second Edition Peter A. Carter
Page
3
Expert Scripting and Automation for SQL Server DBAs: Amplifying Productivity Through Automation, Second Edition ISBN-13 (pbk): 979-8-8688-1150-0 ISBN-13 (electronic): 979-8-8688-1151-7 https://doi.org/10.1007/979-8-8688-1151-7 Copyright © 2025 by Peter A. Carter This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Shaul Elson Desk Editor: Laura Berendson Editorial Project Manager: Gryffin Winkler Cover designed by eStudioCalamar Cover Photo by Alexander Psiuk on Unsplash Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza, New York, NY 10004. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a Delaware LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub. For more detailed information, please visit https://www.apress.com/gp/services/ source-code. If disposing of this product, please recycle the paper Peter A. Carter London, UK
Page
4
This book is dedicated to Terri – the world’s greatest pig farmer.
Page
5
(This page has no text content)
Page
6
(This page has no text content)
Page
7
(This page has no text content)
Page
8
(This page has no text content)
Page
9
(This page has no text content)
Page
10
xi About the Author Peter A. Carter is a SQL Server expert with two decades of experience developing, administering, and architecting SQL Server platforms and data-tier applications. Peter was awarded an MCC by Microsoft in 2011 and currently leads a team of platform engineers within a major UK company. His passion for SQL Server shows through in everything he does, and he has written a number of SQL Server books for Apress, including Pro SQL Server 2022 Administration, SQL Server Advanced Data Types, Securing SQL Server, and SQL Server 2019 AlwaysOn.
Page
11
xiii About the Technical Reviewer Adam Polka is a Senior Platform Engineer working in a polycloud environment, with particular interest in automation, systems integration, and observability. He has worked on a number of projects as a lead engineer, and within his team he is involved in code quality control and training. He lives halfway up a mountain with his family, and when he is not sitting in front of his screens, he can be found renovating an old house, walking the dog, skiing, or trying to avoid being beaten up at karate.
Page
12
xv I want to offer a heartfelt thank-you to Adam Polka for an incredibly detailed technical review of this book. Throughout all of the years that we have collaborated, you have always demonstrated not just a one-in-a-million technical mind and the sharpest eye for detail I have ever met but also remarkable dedication and work ethic. Your review of this book was no different, and your efforts have had a big impact, as always. Acknowledgments
Page
13
1 © Peter A. Carter 2025 P. A. Carter, Expert Scripting and Automation for SQL Server DBAs, https://doi.org/10.1007/979-8-8688-1151-7_1 CHAPTER 1 T-SQL Techniques for DBAs In this chapter, we will discuss some fundamental techniques in T-SQL which will become useful in later chapters, when we discuss how to use PowerShell to automate your SQL Server enterprise. Specifically, we will discuss how to use the APPLY operator to call a function against rows within a result set. It is important to understand this technique in Chapter 8, when we begin to look at metadata-driven automation. We will then look at how XML (eXtensible Markup Language) and how the native XML data type can be harnessed by SQL Server DBAs. It is critical for DBAs to have a handle on the use of XML, due to the volume of information, such as Query Plans, that is stored in this format. Understanding basic XML in SQL Server is also key to understanding how to loop efficiently. To this end, we will explore how to efficiently iterate through multiple objects, using an XML technique. We will compare its efficiency to that of a traditional cursor. This is important, because although looping in PowerShell is always more efficient, there are times where the optimal approach to automation will involve multiple layers of looping, through the PowerShell and T-SQL layers. This will become apparent in Chapter 8, but the technique is used in multiple chapters throughout the book. Using the APPLY Operator The T-SQL APPLY operator will be used in Chapter 8, when we explore metadata-driven automation. It allows you to call a table-valued function against every row in a result set returned by a query. There are two variations of the APPLY operator: CROSS APPLY and OUTER APPLY. When CROSS APPLY is used, the query will only return rows in which a result set has been produced by the table-valued function. When OUTER APPLY is used,
Page
14
2 no filter will be applied to the result set, and where no result is returned by the table- valued function, NULL will be returned, in each column of the table-valued function. The APPLY operator can be very useful to DBAs when they are retrieving metadata from dynamic management views (DMV) and functions (DMF). For example, the query in Listing 1-1 will return a list of all sessions, detailing the session ID, the login time, the login name, and whether the process is a user or system process. It will then use OUTER APPLY to run the sys.dm_exec_sql_text dynamic management function against each row. This function returns a column called text, which is the SQL statement associated with the SQL handle in the sql_handle column in the sys.dm_exec_requests dynamic management view. Note The text column that is referenced in the SELECT list is the column returned by the sys.dm_exec_sql_text DMF. Listing 1-1. Using OUTER APPLY SELECT s.session_id , s.login_time , s.login_name , s.is_user_process , dest.[text] FROM sys.dm_exec_sessions s INNER JOIN sys.dm_exec_requests r ON r.session_id = s.session_id OUTER APPLY sys.dm_exec_sql_text(sql_handle) dest; You will notice that this query returns many rows in this result set with NULL values for the text column. This is because they are system processes, running background tasks for SQL Server, such as the Lazy Writer and the Ghost Cleanup Task. ChapTer 1 T-SQL TeChniQueS For DBaS
Page
15
3 Tip We can be sure that they are system processes because of the is_user_ process flag. We will not rely on the session iD being less than 50. The assertion that all system processes have a session iD of less than 50 is widely believed, but also a fallacy, because there can potentially be more than 50 system sessions running in parallel. If we use the CROSS APPLY operator for the same query, as shown in Listing 1-2, the only rows that will be returned will be rows where the result of applying the table-valued function is not NULL. Listing 1-2. Using APPLY SELECT s.session_id , s.login_time , s.login_name , [text] , s.is_user_process FROM sys.dm_exec_sessions s INNER JOIN sys.dm_exec_requests r ON r.session_id = s.session_id CROSS APPLY sys.dm_exec_sql_text(sql_handle) ; XML for DBAs XML (eXtensible Markup Language) is used extensively as a data format within SQL Server, for storing data that is critical for the successful DBA, such as Extended Events data, query plan data, and collector items, used by MDW (Management Data Warehouse). Therefore, it is important that DBAs have a grasp of the XML format and how to work with it, within SQL Server. Understanding how to use XML in SQL Server is also important to understanding how to loop efficiently. This will be discussed in the “Efficient Looping” section of this chapter. The following sections will provide an overview of the XML data type, how to generate XML documents, and how to query XML data. ChapTer 1 T-SQL TeChniQueS For DBaS
Page
16
4 Understanding XML XML is a markup language, similar to HTML, which was designed for the purpose of storing and transporting data. Like HTML, XML uses tags. Unlike HTML, however, these tags are not predefined. Instead, they are defined by the document author. An XML document has a tree structure, beginning with a root node and containing child nodes (also known as child elements). Each element can contain data but also attributes. Each attribute can contain information that describes the element. For example, imagine that you require details of sales orders to be stored in XML format. It would be sensible to assume that each sales order would be stored in a separate element within the document. But what about sales order properties, such as order date, customer ID, product IDs, quantities, and prices? These pieces of information could either be stored as child elements of the sales order element, or they could be stored as attributes of the sales order element. There are no set rules for when you should use child elements or attributes to describe properties of an element. This choice is at the discretion of the document author. The XML document in Listing 1-3 provides a sample XML document, which holds the details of sales orders for a fictional organization. Listing 1-3. Sales Orders Stored in an XML Document <SalesOrders> <SalesOrder CustomerID="29825" OrderDate="2024-05-31T00:00:00" SalesOrderID="43659"> <LineItem UnitPrice="2024.9940" OrderQty="1" /> <LineItem UnitPrice="2024.9940" OrderQty="3" /> <LineItem UnitPrice="2024.9940" OrderQty="1" /> <LineItem UnitPrice="2039.9940" OrderQty="1" /> <LineItem UnitPrice="2039.9940" OrderQty="1" /> </SalesOrder> <SalesOrder CustomerID="29825" OrderDate="2024-08-31T00:00:00" SalesOrderID="44305"> <LineItem UnitPrice="2024.9940" OrderQty="2" /> <LineItem UnitPrice="2039.9940" OrderQty="2" /> <LineItem UnitPrice="20.1865" OrderQty="2" /> </SalesOrder> </SalesOrders> ChapTer 1 T-SQL TeChniQueS For DBaS
Page
17
5 There are several things to note when looking at this XML document. First, elements (or nodes) begin with the opening tag, which is the element name, encapsulated within angle brackets. They end with the closing tag, which is the element name preceded by a backslash and enclosed in angle brackets. Any elements which fall between the opening and closing tags of another node are child elements of that node. Attribute values are enclosed in double quotation marks and reside within the opening tag of an element. For example, SalesOrderID is an attribute of the <SalesOrder> element. It is acceptable to have repeating elements. You can see that <SalesOrder> is a repeating element, as two separate sales orders are stored in this XML document. The <SalesOrders> element is the document’s root element and is the only element that is not allowed to be complex. This means that it cannot have attributes and cannot be repeating. Attributes can never repeat within an element. Therefore, if you require an attribute to repeat, you should use a nested element as opposed to an attribute. The format of an XML document can be defined by an XSD (XML Schema Definition) schema. An XSD schema will define the document’s structure, including data types, if complex types (complex elements) are allowed, and how many times an element must occur (or is limited to occurring) within a document. It also defines the sequence of elements. A full description of XSD schemas can be found at en.wikipedia.org/wiki/ XML_Schema_(W3C). Tip an XML document requires a root element in order to be “well-formed.” an XML document without a root element is known as an XML fragment. This is important, because it is not possible to bind an XML fragment to an XSD schema. This means that the structure of the document, including data types, cannot be enforced. So how does this map to DBA data? Well, take Listing 1-4 as an example. This shows part of an XML execution plan, which was produced by selecting the database_id column from msdb.dbo.suspect_pages. ChapTer 1 T-SQL TeChniQueS For DBaS
Page
18
6 Listing 1-4. Partial XML Execution Plan <?xml version="1.0" encoding="utf-16"?> <ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.539" Build="15.0.2000.5" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/ showplan"> <BatchSequence> <Batch> <Statements> <StmtSimple StatementCompId="1" StatementEstRows="1" StatementId="1" StatementOptmLevel="TRIVIAL" CardinalityEstimationModelVersion="150" StatementSubTreeCost="0.0032831" StatementText="SELECT database_ id
FROM msdb.dbo.suspect_pages" StatementType="SELECT" QueryHash="0xF3102DE40CF46DEF" QueryPlanHash="0x7549ED4EC00A137C" RetrievedFromCache="true" SecurityPolicyApplied="false"> <StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_ WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" /> <QueryPlan DegreeOfParallelism="0" NonParallelPlanReason="MaxD OPSetToOne" CachedPlanSize="16" CompileTime="0" CompileCPU="0" CompileMemory="88"> <MemoryGrantInfo SerialRequiredMemory="0" SerialDesiredMemory="0" GrantedMemory="0" MaxUsedMemory="0" /> <OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="245509" EstimatedPagesCached="15344" EstimatedAvailableDegreeOfParallelism="1" MaxCompileMemory="407864" /> <QueryTimeStats CpuTime="0" ElapsedTime="0" /> <RelOp AvgRowSize="11" EstimateCPU="7.96E-05" EstimateIO="0.0032035" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" ChapTer 1 T-SQL TeChniQueS For DBaS
Page
19
7 EstimateRows="1" EstimatedRowsRead="1" LogicalOp="Table Scan" NodeId="0" Parallel="false" PhysicalOp="Table Scan" EstimatedTotalSubtreeCost="0.0032831" TableCardinality="0"> <OutputList> <ColumnReference Database="[msdb]" Schema="[dbo]" Table="[suspect_pages]" Column="database_id" /> </OutputList> <RunTimeInformation> <RunTimeCountersPerThread Thread="0" ActualRows="0" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="0" ActualCPUms="0" ActualScans="1" ActualLogicalReads="0" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" /> </RunTimeInformation> <TableScan Ordered="false" ForcedIndex="false" ForceScan="false" NoExpandHint="false" Storage="RowStore"> <DefinedValues> <DefinedValue> <ColumnReference Database="[msdb]" Schema="[dbo]" Table="[suspect_pages]" Column="database_id" /> </DefinedValue> </DefinedValues> <Object Database="[msdb]" Schema="[dbo]" Table="[suspect_ pages]" IndexKind="Heap" Storage="RowStore" /> </TableScan> </RelOp> </QueryPlan> </StmtSimple> </Statements> </Batch> </BatchSequence> </ShowPlanXML> ChapTer 1 T-SQL TeChniQueS For DBaS
Page
20
8 While the execution plan is very simple, you can see that the author used nested elements, all of which have many attributes associated with them. Because the document conforms to a schema (typed), the root element, in this case <ShowPlanXML>, defines its schema association. Converting Result Sets to XML T-SQL allows you to convert relational result sets into XML, by using the FOR XML clause in your SELECT statement. There are four modes that can be used with the FOR XML clause: FOR XML RAW, FOR XML AUTO, FOR XML PATH, and FOR XML EXPLICIT. The following sections will briefly discuss how the FOR XML clause works in RAW mode and AUTO mode, but we will mainly focus on PATH mode. EXPLICIT mode is beyond the scope of this book, as its functionality is very similar to PATH mode but is far more complex and, although helpful in some development scenarios, does not often prove useful for DBAs. Using FOR XML RAW The simplest and easiest to understand of the FOR XML modes is FOR XML RAW. This mode will transform each row in a relational result set into an element within a flat XML document. Consider the query in Listing 1-5, which extracts data from sys.dm_exec_ sessions and sys.dm_exec_requests. Listing 1-5. Extract Session Information SELECT ExecSessions.session_id , ExecSessions.login_name , ExecSessions.host_name , ExecSessions.status , ExecRequests.command , DB_NAME(ExecRequests.database_id) AS DatabaseName , ExecRequests.last_wait_type FROM sys.dm_exec_sessions ExecSessions INNER JOIN sys.dm_exec_requests ExecRequests ON ExecSessions.session_id = ExecRequests.session_id WHERE ExecSessions.is_user_process = 1 ChapTer 1 T-SQL TeChniQueS For DBaS