Artificial intelligence with SQL Server

D

ata is the business asset for any organisation which is audited and protected. To gain in their business, it is become very urgent for every organization to choose few good predictive data models and validates them using test data before figuring out an operationalization plan for the model to be deployed to production so that applications can consume it.

It is true the data and artificial intelligence is growing with each other and we have to do agree that database platforms were just using for the fundamental operations on data in the from of queries or CRUD operations as well as some basic computation routines not more than that.  With built-in R and Python support in SQL Server 2017 release, SQL Server is in a unique position to fuel innovations that database professionals and developers can co-create with the data science and AI communities. The possibilities are endless.

In the current era, Machine Learning is very fast growing field because it is used when searching the web, placing ads, credit scoring, stock trading and for many other applications. With the help of new technology called Artificial intelligence (AI), we are looking at an incredibly exciting time for marketing and customer experience, with huge benefits for the consumer and faster than real-time customer service.

AI services will transform how we all interact with media; by understanding our needs ahead of the game, it will change our lives. By entering in conversation directly with a company, and receiving a directly personalised service in return, we will feel that we are being really taken care of as individuals.

Artificial intelligence is reliable on the huge data which is coming from the heterogeneous sources and we never denied that data movement is very costly for any organisation. By doing data science and AI where the data resides, there are many benefits. These include being able to take advantage of the enterprise-grade performance, scale, security and reliability that you’ve come to expect from SQL Server over the years. The most important benefit is that we can eliminate the need for expensive data movement.

By encapsulating the machine learning and AI models as part of the SQL Server stored procedure, it lets SQL Server serve AI with the data. There are other advantages for using stored procedures for operationalizing machine learning and AI (ML/AI).

That why, Microsoft development team has, arguably, built the most complete Machine Intelligence (MI) technology suite in the market. The list of Microsoft’s MI technologies includes advanced platforms such as Azure ML or R Server, Artificial intelligence APIs such as Microsoft Cognitive Services, data visualisation tools such as PowerBI or even vertical solutions included in the Cortana Intelligence Suite. 

In addition, if the data science project involves working with spatial data, non-structured,  temporal data or semi-structured data, we can leverage SQL Server capabilities that let us do this efficiently which takes important steps to bring new MI capabilities to the traditional database platform. 

Most companies already possess reams of data that is not being used; they need to put it to work. Data, analysed by MI/AI, can be used to develop products and services based on patterns and trends of customer behaviour and preferences. In future, we are going to see MI/AI algorithms becoming as common as data access operations in database servers. Microsoft is in a unique position to lead this new trend but we should expect similar moves by competitors such as Oracle, IBM or newcomers such as MongoDB or Couchbase.

For businesses that are small or failing in the current environment where big players take all, AI/MI could provide a real advantage over the competition – these companies need to engage with it now, and fast, to make the best of this advantage.

Conclusion

SQL Server supports Python and R which will allow developers to implement MI/AI models that natively process data stored in SQL Server databases. Those MI/AI models can be directly persisted in the underlying database servers and scaled as part of SQL Server clusters. More importantly, developers will have access to these capabilities using the familiar SQL Server tool set.

What is Engineering data

Engineering data is the foundation for all of the recent, current, and future data hypes: machine learning, deep learning, big data, data science, etc. The success and adoption of these hypes is predicated on data being structured properly and available. However, when customers (internal and external) are not clear on what their expectations are and/or the big picture of what they are trying to use the data for, data engineers are often blamed. 

Machine learning is a method used to devise complex models and algorithms that lend themselves to prediction; These analytical models allow researchers, data scientists, engineers, and analysts to produce reliable, repeatable decisions and results.

Communication is key! Couple of points, "big data" has always been there, just ask Statisticians, as for the data science hype, now every one calls themselves data scientist, somehow knowing SQL makes a person data scientist now. Data Science graduates on a daily basis and many are lacking basic analytical skills and believe Data Science and Data Analysis are all about having basic end-user level knowledge of a new fancy software. It is a hype for a fact.

We can have the best people using the data for analytics or modeling, but if we don’t have people that know how to build the systems to make our data available in a consistent reliable manner then you will just be part of the hype. Having multiple data science teams leads to friction between the teams. While they, mainly concerned about product delivery, has an open door policy across the Enterprise, the other, more concerned about, well we don't know what, treats everyone else with a sense of superiority and thinks that their work is super secret somehow.

The reason for it's crashing is most likely because the software you are using is not a sever software/service, eg. if you use excel everything is processed in memory(like MS products is doing it all the time) if you had a db then it had been paging to disk when then the assign memory get full (short cut explanation).  

My understanding of Data scientist is they try to hard code/program different scenarios to get an answer.  If you know some about philosophy you cans tart there. You say what if condition 1 =1, then they add different condition with different variables.  It's not actually that complicated from my understanding. 

The data science hype now is focused on AI. After the dust settles they will realize that not all big data and data science is what's going on in Google or Facebook. Corporate needs another big data and data science. They need to make sense of their own data and solve their own problems using whatever techniques. I believe Optimization and statistical models are more important than AI for most of corp-orates. They can buy the hard stuff like speech and text analytics from Google or Amazon or anyone of this scale and capacity. Yet they have to retain a team to solve their specific problems using science.

That the industry does not want to reveal the pressing questions they want to get answered - obviously due to competition and conflict of interest. There are some good concepts behind data science and modeling (both mathematical and statistical) lies at the heart of obtaining directed insights. But yes, academia-industry partnership is very much lacking, which has the potential to make data science a hope or a hype.