In-memory Indexes: Every memory-optimized
table must have at least one index. For durable memory-optimized tables, a
unique index is required to uniquely identify a row when processing
transaction log records change during Page 5 recovery.
They reside only in-memory and not stored in checkpoint
files nor are any changes to the indexes logged.
They maintained automatically during all modification
operations on memory-optimized tables, just like B-tree indexes on disk-based
tables, Note: If SQL Server
restarts, the indexes on the memory-optimized tables are rebuilt as the data
are streamed into memory
|
Empowering you to easily learn and integrate Microsoft and Generative AI solutions.
Thursday, June 30, 2016
SQL - ColumnStore Index on In-Memory Tables
SQL - R Services in SQL Server 2016
- Scale and performance: While R is open-source and free, R packages run single-threaded and can consequently only consume datasets that fit into available memory on the local machine. This memory constraint becomes a critical drawback as data sizes grow or when wishing to speed up execution.
- Integrating with applications: R code can be easily integrated with business applications in production but many enterprises end-up translating the R code into different languages in order to integrate it. This process is typically cumbersome, costly and error-prone and doesn’t support agile innovation. Enterprises need a simple way for applications to retrieve predictions, scores and plots, schedule R jobs and integrate it with existing application logic.
- Data movement: Moving data out of the database is best avoided when possible. Having to transfer data for analysis on a separate machine running R can present security headaches and latency issues when data volumes grow. It also tends to create data fragmentation because different people in the organization find themselves working on different versions of the data.
Apart from this; we have to facility to work on the large datasets which provides the multiple benefits-
- It brings the R applications to the production through the most familiar T-SQL interface. This is because most of the developers are aware about the T-SQL and they can easily integrate R with their T-SQL code and use it through the SQL Server.
- It avoids the need of data movement between our data and R computation and R computation will be running on the same server as the database in DB analytics. It facilitates to work on the same copy of the data in the data store.
- It provides a superior performance and scale for common operations, over open source R while allowing the use of any R package. It allows scaling for large datasets, far exceeding the available memory on the machine.
The primary benefit is secure, scalable analytics: R scripts run in isolated sessions managed by SQL Server's extensibility framework, preventing data leakage and leveraging parallel processing across server cores. Common use cases include data cleaning, feature engineering, model training, and predictive scoring—all without exporting data over networks.
History and Evolution
- SQL Server 2016: Launched as "R Services," focusing solely on R integration. It included Microsoft R Open (an enhanced open-source R distribution) and packages like RevoScaleR for scalable analytics.
- SQL Server 2017: Renamed to "Machine Learning Services" to add Python support. R capabilities were enhanced with newer runtimes and additional packages.
- SQL Server 2019–2022: Continued refinements, including support for more R packages and better integration with Azure. Microsoft R Open was retired, shifting to standard open-source R distributions.
- As of 2025: No major deprecations for R in Machine Learning Services, but starting with SQL Server 2022, R runtimes are not bundled in setup—users must install custom runtimes post-installation. Legacy packages like MicrosoftML (R), olapR, and sqlrutils are limited to SQL Server 2016–2019.
Key Features for R
Machine Learning Services provides a rich set of tools for R users:
| Feature | Description |
|---|---|
| In-Database Execution | Run R scripts via T-SQL using sp_execute_external_script, keeping data secure and reducing latency. |
| Scalable Packages | RevoScaleR for distributed processing of large datasets; supports rx functions for modeling and visualization. |
| Pre-Installed Packages | Includes base R, common open-source libraries (e.g., ggplot2, dplyr), and Microsoft-specific ones (version-dependent). Additional packages can be installed via R tools. |
| Parallel Processing | Automatic use of multiple cores for computations; hint with @parallel = 1 for certain operations. |
| Integration Tools | sqlrutils (older versions) for embedding R in stored procedures; support for MDX queries via olapR (deprecated in newer versions). |
R runtime versions vary by SQL Server edition:
| SQL Server Version | R Runtime Version |
|---|---|
| 2022 (RTM+) | 4.2.0 |
| 2019 (RTM+) | 3.5.2 |
| 2017 (CU22+) | 3.3.3 / 3.5.2 |
| 2017 (RTM–CU21) | 3.3.3 |
| 2016 | 3.2.2–3.5.3 (varies by update) |
Installation
Installation depends on your SQL Server version. Machine Learning Services must be selected during SQL Server setup (or added later via Add Features).
- For SQL Server 2016–2019 (Windows):
- Run SQL Server setup, select "R Services" (or "Machine Learning Services" in 2017+).
- Post-install, enable the feature on the instance: EXEC sp_configure 'external scripts enabled', 1; RECONFIGURE;
- Verify: EXEC sp_execute_external_script @language=N'R', @script=N'print("R is ready")';
- For SQL Server 2022+ (Windows/Linux):
- Install base SQL Server without runtimes.
- Download and install R runtime (e.g., via Rtools or custom packages) from CRAN or Microsoft sources.
- Register the runtime: Use sqlmlutils Python package or manual configuration.
- Enable external scripts as above.
- On Linux: Use mssql-mlservices-config tool.
For detailed steps, refer to official docs for your platform. Prerequisites include admin rights and sufficient disk space (R runtime ~1–2 GB).
References: Microsoft
Saturday, June 25, 2016
SQL - Real Time Operational Analytics
Note: Real-time operational analytics does not replace the need for a separate data warehouse when business needs to integrate data from multiple sources before running the analytics workload.
|
Benifits:
|
Still working on.....
References: Microsoft
Thursday, June 23, 2016
SQL - Amazing features in SQL Server 2016
Mobile BI apps provide live, interactive, mobile access to our important business information and interact with our data easily in a touch-optimized experience with the Power BI native Windows, iOS, and Android apps.
The incorporation of R is important for data scientists because they won't have to take code out of the database in order to run the R programming language. R allows the user to run queries on server data.
SQL Server R Services is a set of SQL Server tools and technologies that offer superior performance, security, reliability and manageability. In SQL Server 2016, SQL Server R Services provides both server and client components.
Open Source R packages run single-threaded and can therefore only consume datasets that fit into available memory on the local machine.
R Services support in-database analytics allowing us to keep the data in SQL Server 2016 and have applications call R scripts via a T-SQL system stored procedure.
The main thing is that to improve the performance, you need to define the limits where you could say that this data could not be available for NCCI for HOT/WARM (Predicate). The most important point in SQL Server 2016, NCCI and CCI will share the same code means same performances is applicable for the both indexes.

At a glance
Saturday, June 18, 2016
SQL- String_Split() function
|
Syntax
STRING_SPLIT
(String, Separator)
Arguments
String: Is an expression of any character type (i.e.
nvarchar, varchar, nchar or char).
Separator: Is a single character expression of any
character type (e.g. nvarchar(1), varchar(1), nchar(1) or char(1)) that is
used as separator for concatenated strings.
Return Types: Returns a
single-column table with fragments. The name of the column is value. The
length of the return type is the same as the length of the string argument.
|
- Default return data type is varchar and it will return nvarchar if any of the input arguments are either nvarchar or nchar.
- It will return an empty table If the input string is NULL.
- It will return empty string if there is nothing between separators. Condition RTRIM(value) <> '' will remove empty tokens.
- STRING_SPLIT requires at least compatibility mode 130.
- If your database compatibility level is lower than 130, SQL Server will not be able to find and execute STRING_SPLIT function.
- We can change the compatibility level of database as following: ALTER DATABASE DatabaseName SET COMPATIBILITY_LEVEL = 130
|
----Local
Variables
Declare @StringValue Varchar(200)
Set @StringValue='Ryan Arjun, Kimmy
Wany, Rosy Gray,,Johnson Blue'
---- Separator
Variable
Declare @Separator char(1)=','
---- Using String
Split function
Select Value
from string_split(@StringValue,@Separator)
STRING_SPLIT will return empty string if there is nothing
between separator.
|
|
----Local
Variables
Declare @StringValue Varchar(200)
Set @StringValue='Ryan Arjun, Kimmy
Wany, Rosy Gray,,Johnson Blue'
---- Separator
Variable
Declare @Separator char(1)=','
---- Using String
Split function
Select Value
from string_split(@StringValue,@Separator)
---- remove empty strings from output
where RTRIM(value) <> '';
|
|
----Declare table
variable
Declare @ProductTable Table
(
ProdId Int,
ProductName varchar(50),
Tags Varchar(200)
)
---- Fill Data
into table variable
Insert into @ProductTable (ProdId, ProductName, Tags)
Values (1, 'Full-Finger Gloves', 'clothing,road,touring,bike'),
(2,'LL Headset','bike, cycle'),
(3,'HL Mountain Frame','bike,mountain,weather')
---- Pull values
from table Variable
SELECT ProdId, ProductName, Tags
FROM
@ProductTable
---- Separator
Variable
Declare @Separator char(1)=','
---- Using String
Split function
SELECT ProdId, ProductName, Value
FROM
@ProductTable
CROSS APPLY
STRING_SPLIT(Tags, @Separator);
----- Aggregation
by values
SELECT value as tag, COUNT(*) AS [Products]
FROM
@ProductTable
CROSS APPLY STRING_SPLIT(Tags, ',')
GROUP BY value HAVING COUNT(*) > 2
ORDER BY COUNT(*) DESC;
----shows the
number of products per each tag, ordered by number of products, and to filter
only the tags with more than 2 products.
----- Search by
tag value: To find products with a single tag
SELECT ProdId, ProductName, Tags
FROM
@ProductTable
WHERE 'bike'
IN (SELECT value FROM STRING_SPLIT(Tags, ','));
----- Search by
tag value: To find products with a multiple tags
SELECT ProdId, ProductName, Tags
FROM
@ProductTable
WHERE EXISTS (SELECT *
FROM STRING_SPLIT(Tags, ',')
WHERE value IN ('clothing', 'mountain'));
|













