[hemmerling] Data Processing 5/7 - Business Intelligence, Data Mining


Conferences, Seminars, Trainings




Free ETL Software

Free Tools


Free BI Software

Free Data Warehouse Solutions

Free BI Tools

Free BI Frameworks

Data Cleanup

  • The OpenSource OpenRefine, GitHub "OpenRefine" ( formerly: “Google Refine” ) - “A powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data”.

Machine Learning

Important Machine Learning Tools according to "Developer Economics - National Trends Survey", 2016-04
  • IBM Watson
  • BigML
  • SAS Enterprise Miner
  • R platform
  • Python machine learning libraries
  • Google Prediction API
  • WEKA Machine Learning Workbench
  • Microsoft Azure Machine Learning
  • AWS Machine Learning
  • TensorFlow
  • PredictionIO
  • MATLAB or Octave
  • DMTK
  • IBM SPSS Modeler
  • Apache Spark
  • RapidMiner
  • The OpenSource library Tensorflow, GitHub "tensorflow" - “TensorFlow is an Open Source Software Library for Machine Intelligence”, “Open source software library for numerical computation using data flow graphs”.
  • GitHub "google/skflow" - “Simplified interface for TensorFlow (mimicking Scikit Learn)”.

Microsoft BI Tools

The free standalone Microsoft Power BI

The Tool
  • “Microsoft Power BI” is a dashboard tool, not a reporting tool!
    • It is powered by “Microsoft SQL Server Analysis Services (SSAS)”.
    • It is now easy to catch data from HTML pages ( e.g. data tables of Wikipedia ).
    • Data visualisation:
      • Built-in graphics ( by a graphics engine similar / identical to that known from Micrsoft Excel, Microsoft SQL Server Reporting Services and Microsoft SQL Server Analysis Services ).
      • There is an API by which third parties may offer own visualisations.
    • Indeed with Microsoft Excel 2016, you may create the same dashboards, but you can´t publish it on the web.
  • You may save “Microsoft Power BI Desktop” as in single ”.pbix” files. Experts told me, that for saving a project, the software needs 3 times of the RAM memory as needed for the loaded data. So it might be that you may load big data, but are not able to save it. In general, this might not be a problem with the 32-bit edition ( on Windows PCs with 2 or 3 GByte of RAM ), but also with the 64-bit edition ( i.e. a project may be saved on a Win64 computer with 16 GB RAM, but not on a Win64 computer with 4 GB RAM ).
  • You might create test cases ( e.g. at business intelligence trainings ) with “Microsoft Power BI”, as alternative to NBI.
  • “Microsoft Power BI Desktop” also supports the “M” and “DAX” query languages, by the “Advanced Editor” ( “Home / Edit Queries ( = Query Editor ) / Advanced Editor” ).
  • If you install “Microsoft R” prior to “Microsoft Power BI Desktop”, you may use “R” for data processing and graphical reporting → See Mathematical Engineering.
  • I was told by experts, in 2016-03:
    • Now legacy versions of “Microsoft Power BI Desktop” were able to process 10.000 datasets for visualisation.
    • Current versions of “Microsoft Power BI Desktop” were able to process 10.0000 datasets for visualisation.
    • With “R” called by “Microsoft Power BI Desktop”, you may visualize 150.000 datasets.

Commercial Editions of Microsoft SQL Server with BI Support

Power BI Gateway - Personal

The Tool
  • To get a BI report, connect with a client ( Internet browser, smartphone app,..) to the “Microsoft Power BI” cloudservice. The basic “Power BI” account is free.
  • By this, you may access any cloud document.
  • The cloud servce may access a registered onpremise Windows server at your ( the datacenter of your ) company, if the computer runs a free “Power BI Gateway - Personal” service, by a VPN connection.
    • Experts told me, that the free VPN service shipped with Windows is not under development for some time. Therefore it isn´t very advisable to use it in production environments.

Free Addons for the commercial Microsoft Excel


Query and Data Modeling Languages

Azure Log Analytics Query Language
  • See on this page.
Analysis Services Scripting Language ( ASS )
Data Analysis Expressions ( DAX ) - Data Modeling Language
"M" Language ( "Power Query Formula Language" / Microsoft Power Query for Excel Formula Language" )
MultiDimensional eXpressions ( MDX )
Tabular Modeling Scripting Language ( TMSL )


Data Science Platform by Joshua Görner

"TWDE Datalab" by ThoughtWorks

Some other commercial BI Software



  • The online cloud based predictive analysis service BigML - “Machine Learning for everyone. Easily add data-driven decisions and predictive power to your company”, “NOW FREE. Unlimited tasks ( up to 16 MB/task )”.

EdgeImpulse Inc.

Google Colab ( Colaboratory )

  • The free online service Google Colab ( Colaboratory ).
  • A free Jupyter Notebook server, most packages of Anaconda are installed, but not 100% compatible ( different version of Python packages, few missing packages ).
  • User experience of Google Drive users ( so you must be logged into Google Drive by your Google account ):
    • Create new Jupyter cells.
      • Button ”+Code” and menu option “Insert / Code cell” create a new code cell, while ”+Text” and menu option “Insert / Text cell” create a new text cell, at the point of the Jupyter notebook, where your cursor is, i.e. where you have hit your mouse by mouse-leftlick.
      • In local Jupyter notebooks, there is a button of the extra button bar to provide this. This button bar is not available with Google Colab.
    • You may create a copy of a Jupyter notebook put on Google Docs, by “File / Save a copy in Drive”.
    • But its most easy to transfer additional data files by downloading them on your local terminal ( PC with browser ), and to upload it into the copied Jupyter Notebook.
    • You might use “Files / Mount Drive” to create the code
      from google.colab import drive

Microsoft KUSTO

The Service

Azure Log Analytics Query Language



The Service




Complex event processing, Streaming Analytics

Realtime Data Warehouse

FIWARE "Orion ( Context Broker )"

Google Cloud Stream Analytics Solution

Microsoft Technologies for Realtime Data Warehouses

Microsoft Azure Stream Analytics

Microsoft SQL Server - StreamInsight

Microsoft KUSTO

  • See on this page.


Data Analysis & Dashboards

Full Text Search

Artificial Intelligence

Webcasts. Webinars




Forums, Newsgroups

When this document changes ! Site Navigation ( My Business ! My Topics ! Imprint / Contact ! Privacy Policy ! Keyword Index ! ! Google+ Publisher "hemmerling" )

en/bintelligence.html.txt · Last modified: 2022/10/02 20:51 (external edit) · []
Recent changes RSS feed Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki