Infragistics JQuery controls

RapidMiner tutorial: How to explore correlations in your data to discover the relevance of attributes

What is correlation? From wikipedia In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence. In laymans terms, correlation is a relationships between data attributes.  For a quick refresher, in data mining, a dataset is made up of different attributes.  We use these attributes to classify or predict a label.  Some attributes have more "meaning" or influence over the label's value.  As you can imagine, if you can determine the influence that specific attributes have over your data, you are in a better position to build a classification model because you will know which attributes you should focus on when building your model.   In this example, I will use the kaggle.com Titanic datamining challenge dataset.  This post will not uncover any information that is not readily available in the tutorial posted on kaggle.com. Here are two screenshots.  The first screenshot will show you some statistics about the dataset.  The second screenshot will show a sample of the data. Meta data view of the Titanic data mining challenge Training dataset A data view of the dataset The correlation matrix First start by importing the Titanic training dataset into RapidMiner.  You can use Read From CSV, Read From Excel, or Read from Database to achieve this step.  Next, search for the "Correlation Matrix" operator and drag it onto the process surface.  Connect the Titanic training dataset output port to the Correlation Matrix operator's input example port.  Your process should look like this.   Now run the process and observe the output. You are presented with several different result views.  The first view will be the Correlation Matrix Attribute Weights view.  The Attribute weights view displays the "weight" of each attribute.  The purpose of this tutorial is to explain a different view of the Correlation matrix.  Click on the Correlation Matrix view.  This is a matrix that shows the Correlation Coefficients which is a measure of the strength of the relationship between our attributes.  An easy way to get started with the Correlation matrix is to notice that when an attribute intersects with itself, you have a dark blue cell with the value of 1 which represents the strongest possible value.  This is because any attribute matched with itself is a perfect correlation.  A correlation coefficient value can be positive or negative.  A negative value does not necessarily mean there is less of a relationship between the values represented.  The larger the coefficient in either direction represents a strong relationship between those two attributes.  If we look at the matrix and follow along the top row (survived) we will see the attributes that have the strongest correlation with the label in which we are trying to predict. Just as the kaggle.com tutorial specifies, the attributes with the strongest correlation with the label (survived) are sex(0.295), pclass(0.115), and fare(0.66)  Remember that the value as well as the color will help you to visually identify the stronger correlation between attributes. If you are working with a classification problem, I'm sure you can see how valuable the correlation matrix can be in showing you the relationships between your label and attributes.  Such insights let can provide a great start on where to focus your attention when building your classification model. Thanks for reading and keep your eyes open for my next tutorial! 


Assistant Professor receives $518,434 to apply Machine Learning to network analysis

The University of Illinois at Urbana-Champaign - College of Engineering has awarded $518,434 to Assistant Professor Maxim Raginsky to use to apply Machine Learning techniques to network analysis to try and discover how to make networks more efficient. From the article http://csl.illinois.edu/news/raginsky-receives-career-award-apply-information-theory-machine-learning-problems “The overall design objective is to make sure that the network resources are allocated in a smart way, and each user receives only the data they need without significant waste of bandwidth or power,”  said Raginsky, a member of Illinois' electrical and computer engineering faculty. Raginsky uses ecological monitoring as an example. If someone is tracking a rare bird species in a specific habitat and wants to record how many of these birds fly in and out of the area, it would be a waste of resources to continuously stream video if what the person really wants is just the arrivals and departures of the birds. A big part of the problem is learning to detect events of interest and to reliably communicate only the data describing these events. “So I want to make sure that only the relevant information gets to those who need it, despite the fact that everyone is using the same network and the kinds of information that are relevant to one user are different than the kinds of information that are relevant to somebody else,” Raginsky said. These problems are messy and complex, and there is no hope to come up with an accurate model for all kinds of data being transmitted and received over networks because of the increasing size and complexity of both the networks and the data, Raginsky said. Machine learning offers a variety of tools for extracting predictively relevant information from observations, but to date most of the research on machine learning has not focused on the network aspect and all the resource constraints that it imposes. This project will systematically explore what is and is not possible in these types of large networks with multiple learning agents, specifically identifying the effect of bandwidth limitations, losses, delays and lack of central coordination on the performance of statistical learning algorithms, thus helping develop efficient and robust coding/decoding schemes. The NSF CAREER Award is awarded by the National Science Foundation specifically to “junior faculty members who demonstrate their roles through outstanding research and education,” according to NSF’s website. Raginsky said that because these awards are for 5-year projects, the proposals take a lot of time and effort. “You propose to research something you’re really passionate about, and presumably you want to work on this topic even if it did not get funded,” Raginsky said. “So, when I heard about my proposal being recommended for funding, of course it was a relief. I will have a good time working on this problem.” Raginsky is a member of the Decision and Control group at CSL. I think that this is a wonderful problem domain in which Machine learning can prove useful.  Machine learning is a powerful set of technologies, and we have yet to even scratch the surface of what it can do for human kind.  This goes to show you that there are other great uses besides targeted advertising systems, though that is where most of the jobs are at the moment.  Do you have ay ideas as to some practical applications of Machine learning that have yet to be tested? Please share by leaving a comment.  


Machine learning resources for .NET developers

Greetings friends and welcome to this article on Machine learning libraries for .NET developers.  Machine learning is a hot topic right now and for good reason.  Personally, I haven't been so excited about a technology since my computer used my 2800 baud modem to dial into a BBS over 17 years ago.  The thought that my computer could communicate with another computer was so fascinating to me.  That moment was the very moment that would forever change my life.  I learned a lot about DOS by writing batch scripts and running other programs that allowed me to visit and then run a BBS system.  It eventually lead me to QBasic.  I wanted to learn to write BBS door games and QBasic was included as a part of a standard DOS installation back then. Fast forward 17 years and I'm still in love with computers, programming, and the concept of communication between machines.  The magic never disappeared.  So when i first learned about the concept of Machine learning, I felt like that 13 year old kid again.  The idea that a machine can learn to do things that it has not been programmed to do is now a passion of mine.  The concepts of Machine learning have an extreme learning curve, however, I believe that we as humans can do anything that we put our mind to.  So I began looking around for tutorials on machine learning.  I found many great tutorials and books, however, most of them involved using python.  I have nothing against python.  As a matter of fact, I find it ironic that I started with BASIC and now in this moment of "rebirth" I'm beginning to use python which looks a lot like BASIC in many ways.  The fact of the matter remains, I'm a .NET developer.  I've spent the last 9 years in the .NET framework and I love the technology.  C# is an awesome programming language and it's hard to imagine life without Visual Studio.  What can I say, the IDE has spoiled me. While I scoured the internet looking for tutorials related to Machine learning resources for .NET developers, I wished that there was a one resource that would assist me in my search for resources to help me achieve my goal. Well that's what this article is all about.  In this article, I will introduce you to some .NET libraries that will assist you in your quest to learn about Machine learning. NND Neural Network Designer by Bragisoft The Neural Network Designer project (NND) is a DBMS management system for neural networks that was created by Jan Bogaerts.  The designer application is developed using WPF, and is a user interface which allows you to design your neural network, query the network, create and configure chat bots that are capable of asking questions and learning from your feed back.  The chat bots can even scrape the internet for information to return in their output as well as to use for learning.  The project includes a custom language syntax called NNL (neural network language) that you can use in configuring your machine learning project.  The source code is designed so that the libraries can be used in your own custom applications so you don't have to start from scratch with such a complex set of technologies.  The project is actually an open source project in which I am a part of.  Some of the possibilities offered by this awesome project include predictions, image and pattern recognition, value inspection, memory profiling and much more.  Stop by the Bragisoft NND website and download the application to give it a try.   Screen shots of the neural network designer by Bragisoft A DBMS for neural networks   Mind map rand forrest The chat bot designer and other tools Accord.net Here is a description from the Accord.NET project website  Accord.NET is a framework for scientific computing in .NET. The framework builds upon AForge.NET, an also popular framework for image processing, supplying new tools and libraries. Those libraries encompass a wide range of scientific computing applications, such as statistical data processing, machine learning, pattern recognition, including but not limited to, computer vision and computer audition. The framework offers a large number of probability distributions, hypothesis tests, kernel functions and support for most popular performance measurements techniques.  The most impressive parts of this library has got to be the documentation and sample applications that are distributed with the project.  This makes the library easy to get started using.  I also like the ability to perform operations like Audio processing (beat detection and more), Video processing (easy integration with your web cam, vision capabilities and object recognition).  This is an excellent place to start with approaching Machine learning with the .NET framework.  Here are a two videos that should whet your appetite. Hand writing recognition with Accord.NET   Here is an example of head tracking with Accord.NET (super cool)   AIMLBot Program# AILM Chat bot library AIMLBot (Program#) is a small, fast, standards-compliant yet easily customizable implementation of an AIML (Artificial Intelligence Markup Language) based chatter bot in C#. AIMLBot has been tested on both Microsoft's runtime environment and Mono. Put simply, it will allow you to chat (by entering text) with your computer using natural language.  The project is located here. Math.NET Machine learning algorithms are extremely math heavy.  Math.NET is a library  that can assist with the math that is required to solve machine learning related problems. Math.NET Numerics aims to provide methods and algorithms for numerical computations in science, engineering and every day use. Covered topics include special functions, linear algebra, probability models, random numbers, interpolation, integral transforms and more. DotNumerics DotNumerics is a website dedicated to numerical computing for .NET. DotNumerics includes a Numerical Library for .NET. The library is written in pure C# and has more than 100,000 lines of code with the most advanced algorithms for Linear Algebra, Differential Equations and Optimization problems. The Linear Algebra library includes CSLapack, CSBlas and CSEispack, these libraries are the translation from Fortran to C# of LAPACK, BLAS and EISPACK, respectively. You can find the library here.  ALGLIB ALGLIB is a cross-platform numerical analysis and data processing library. It supports several programming languages (C++, C#, Pascal, VBA) and several operating systems (Windows, Linux, Solaris). ALGLIB features include: Accessing ‘R’ from C#–Lessons learned Here are instructions to use the R statistical framework from within c# ILNumerics You can check out the library at http://www.ilnumerics.net NuML.net http://numl.net A nice site about the basics of machine learning in c# by Seth Juarez . NuML.NET is a machine learning library for .NET developers written by Seth Juarez.  I've recently tried this library and I'm impressed!  Seth has stated publicly that his intention behind the numl.net library is to abstract the scary math away from machine learning to provide tools that are more approachable by software developers and boy did he deliver!  I've been working with this library for a little more than an hour and I've written a prediction app in c#.  You can find his numl.net library source on github. Encog Machine Learning Framework Here is what the official Heaton Research website has to say about Encog: Encog is an advanced machine learning framework that supports a variety of advanced algorithms, as well as support classes to normalize and process data. Machine learning algorithms such as Support Vector Machines, Artificial Neural Networks, Genetic Programming, Bayesian Networks, Hidden Markov Models and Genetic Algorithms are supported. Most Encog training algoritms are multi-threaded and scale well to multicore hardware. Encog can also make use of a GPU to further speed processing time. A GUI based workbench is also provided to help model and train machine learning algorithms. Encog has been in active development since 2008. Encog is available for Java, .Net and C/C++. Jeff Heaton knows a great deal about machine learning algorithms and he's created a wonderful library called Encog.  I was able to write a neural network application that solved the classic XOR problem in 20 minutes after installing the library.  What really amazes me is that he has an Encog Library for JavaScript which includes live samples on his website of Javascript + encog solving problems like the Traveling Salesman Problem and Conway's game of life, all in a browser!  This library can even use your GPU for the heavy lifting if that's your choice.  I would highly recommend that you at least check out his site and download the library to look at the examples.  You can find the Encog library here.    Conclusion This concludes my article on Machine learning resources for the .NET developer.  If you have any suggestions regarding a project that you know of or you are working on related to Machine learning in .NET, please don't hesitate to leave a comment and I will update the article to mention the project.  This article has shown that we as .NET developers have many resources available to us to use to implement Machine learning based solutions.  I appreciate your time in reading this article and I hope you found it useful.  Please subscribe to my RSS feed.  Until next time.. Buddy James


About the author

My name is Buddy James.  I'm a Microsoft Certified Solutions Developer from the Nashville, TN area.  I'm a Software Engineer, an author, a blogger (http://www.refactorthis.net), a mentor, a thought leader, a technologist, a data scientist, and a husband.  I enjoy working with design patterns, data mining, c#, WPF, Silverlight, WinRT, XAML, ASP.NET, python, CouchDB, RavenDB, Hadoop, Android(MonoDroid), iOS (MonoTouch), and Machine Learning. I love technology and I love to develop software, collect data, analyze the data, and learn from the data.  When I'm not coding,  I'm determined to make a difference in the world by using data and machine learning techniques. (follow me at @budbjames).  

Related links

Month List