[hemmerling] Data Processing 5/7 - Business Intelligence, Data Mining

Related pages:

• Data Processing 1/7 - Databases, Webserver, Webservices, SQL, XML.
• Data Processing 2/7 - Descriptions of Data, Data Transfer and Data Protocols.
• Data Processing 3/7 - NoSQL Databases: Not Just SQL.
• Data Processing 4/7 - Database Managment Tools and Database Application Development Tools.
• Data Processing 6/7 - Data Warehouse Modeling.
• Data Processing 7/7 - Machine Learning.
• FIWARE 1/2.
• Data Visualisation with Infographics.
• Mathematical Engineering.
• Open Data.

• Data Science Association.
• Deutsche Gesellschaft für Informations- und Datenqualität e.V. (DGIQ).
• DING e.V. - Deutschsprachige Informatica Nutzer Gruppe ( formerly: “Deutschsprachige Informatica Nutzergruppe” ).
• The Data Warehouse Institute ( TDWI ) US, The Data Warehouse Institute ( TDWI ) Europe / Germany.
  • The paper magazine BI Spektrum - Fachzeitschrift für Business Intelligence und Data Warehousing. Eine Publikation des TDWI Germany e.V..
• Vereinigung Technischer Analysten Deutschlands e.V. ( VTAD ).
  • Book Robert D. Edwards, John Magee, W. H. C. Bassetti: "Technical Analysis of Stock Trends".
  • Book Martin J. Pring: "Technical Analysis Explained. The Successful Investor's Guide to Spotting Investment Trends and Turning Points".
• Meetup "SAP Stammtisch Hannover".

• Data Modeling Zone in Hannover ( 2013-09-23 - 2013-09-24 ).
• Interdisciplinary Center for Applied Machine Learning.
  • Leibniz Universität Hannover, Institut für Kartographie und Geoinformatik "Research".
    • Leibniz Universität Hannover, Institut für Kartographie und Geoinformatik "Big Data and Machine Learning".
      • Leibniz Universität Hannover, Institut für Kartographie und Geoinformatik "Interdisciplinary Center for Applied Machine Learning - ICAML".
  • Interdisciplinary Center for Applied Machine Learning "Events".
• The Data Warehouse Institute ( TDWI ).
  • The Data Warehouse Institute ( TDWI ) Roundtable.
    • TDWI Roundtable "Hannover".
    • TDWI Roundtable "Hamburg".
  • TDWI Konferenz.
    • TDWI Konferenz 2016 - I joined 2016-06-21 the Seminar “Self-Service Analytics mit Qlik” ( “Self-Service Analytics with Qlik” ) by Digital Ratio GmbH.
      • Digital Ratio GmbH "TDWI-Konferenz 2016: Self-Service Analytics mit QlikDigital Ratio GmbH", 2016-06-21.
      • Digital Ratio GmbH "TDWI-Konferenz 2016: Digitale Transformation der BI", 2016-06-22.
  • TDWI Konferenz 2018 - I joined the full conference, with a sponsored ticket.
  • Conference app.
    • Website ""SIGS DATACOM IT EventApp".
    • Google Play "SIGS DATACOM IT EventApp", 2018.
    • username: Your E-Mail, initial password: “tdwi2018”.
  • The online conference TDWI München 2021, 2021-06-21 - 2021-06-23 - I joined the free seminars of the conference.
  • The conference with free tracks TDWI München 2022.
    • TDWI "Registrierung für das freie Programm der TDWI München 2022".
• MicroStrategy Symposium Series.
  • MicroStrategy Symposium - Munich in München, 2018-10-18.
• TDWI Young Guns.
  • LinkedIn "TDWI Young Guns".
  • XING "TDWI Young Guns"
  • Twitter "TDWI EU".
  • Slack "TDWI Young Guns".
  • Events.
    • The free online barcamp TDWI "Oops - we do it again - Data & Analytics BarCamp Vol. 4", 2021-09-16.
      • Sli.Do "Oops - we do it again - Data & Analytics BarCamp Vol. 4".
      • Traveling Salesman.
        • Miro Board "Oops - we do it again - Data & Analytics BarCamp Vol. 4".
        • GitHub "arnimpdm / Get_in_IT_Travelling_Salesman_Problem".
        • Icon Map.
        • Observable "@D3 / Gallery".
      • OikoSoft "How to succeed with your Agile BI project".
    • The free online barcamp TDWI "Irren ist menschlich – Mach ‘was draus Vol. 2 | Data & Analytics F*ckUp Night", 2022-02-10, 18:00.

• Solution Monday.
  • The free online conference DataOps Unleashed, 2022-02-02 by - “live virtual peer-to-peer sessions”.
    • SolutionMonday "DataOps Unleashed 2022".
  • The free online conference Data Agility Day, 2021.
• Global Data Summit in USA ( 2017, 2019 ).

• The OpenSource apatar - connecting data - “Open Source Data Integration & ETL”.
  • apatar - News/Press Releases - “Apatar, the open source leader in data mashup integration”.
  • apatar - Why Apatar?.
    1. Flexible Deployment Options.
    2. No Coding! Visual Job Designer.
    3. No Coding! Transformation Mapper.
  • ApatarForge.
• The OpenSource CloverETL by Javlin a.s. - “Rapid Data Integration Software and Open Source ETL Tool”.
  • CloverETL "Open Source & Community Edition".
  • CloverETL "CloverETL Community Edition".
  • Blog ClovertETL Blog "CloverETL Blog on Data Integration".
  • GitHub "CloverETL".
  • SourceForge "CloverETL", SourceForge "CloverETL" - “Rapid data integration framework”.
• The OpenSource Talend Open Studio.
  • Talendforge.
  • EN.Wikipedia "Talend Open Studio".

• alooma "Open Source ETL Tools Comparison".

• “MariaDB AX” → See Data Processing 1/7 - Databases, Webserver, Webservices, SQL, XML.

• The OpenSource Apache Superset.
  • EN.Wikipedia "Apache Superset".
• The free Ataccama "DQ Analyzer" - “Advanced Data Profiling Tool” for Windows.
• The OpenSource BI reporting tool Eclipse BIRT by Actuate Corporation - The BIRT Company.
  • BIRT Exchange - “Products and Services for BIRT”, “Get the most out of BIRT”.
  • BIRT uses JDBC to access databases. To access an ODBC-only database on Windows, you need a JDBC-ODBC Bridge Driver. The features of the JDBC drivers determine what BIRT can make from the databases, e.g. concerning support of standard and non-standard SQL datatypes ect.
  • The fictive company of the sample database is “Classic Models Inc.”.
  • Actuate Corporation - The BIRT Company "BIRT iHub F-Type. New Free BIRT Server".
  • Commercial competitors: IBM Cognos, Crystal Reports, QlikView.
• The free Jaspersoft Business Intelligence Suite, Community Edition.
  • JasperForge.
• KNIME - Open for Innovation.
  • KNIME "KNIME Software".
    • The OpenSource KNIME "KNIME Analytics Platform - Open, intuitive, integrative data science".
  • KNIME "KNIME Community".
    • KNIME "Community Contributions".
• The free MicroStrategy Reporting Suite for Microsoft Analysis Services.
  • MicroStrategy - Support Site.
  • MicroStrategy - Resource Center.
  • MicroStrategy Deutschland GmbH.
  • Free online trainings MicroStrategy Deutschland GmbH - "Kostenloses Online Training".
  • The free MicroStrategy - "Das MicroStrategy Jump-Start Projekt".
• The free Oracle Warehouse Builder (OWB) 11g.
  • “OWB 11.2 is installed as part of every Oracle Database 11g Release 2 installation”.
  • The OWB is linked to a specific version of Oracle Database. So you can´t use OWB 11.2 with Oracle 12 database. As OWB is discontinued, there will be no newer version available for newer Oracle databases ( e.g. Oracle 12,.. ) .
  • EN.Wikipedia "Oracle Warehouse Builder".
• OW2 Consortium - “Leading open source middleware”.
• QlikTech International AB "QlikView".
  • Software:
    • “QlikView”.
      • The free QlikTech International AB "Download QlikView Personal Edition. Experience the power of Qlik on your personal computer", QlikTech International AB "Seeing is Believing. Downloaden Sie QlikView noch heute!" for Win64 “Qlik lädt Sie dazu ein, sich die QlikView Personal Edition downzuloaden. Diese ist keine Testversion, sondern eine komplette Version unserer leistungsfähigen Business-Discovery-Software”.
      • “QlikView 12” is a local server application for Win64 computers, with a rich client applications for Win64.
      • “QlikView 11” was the latest edition with both Win64 and Win32 server and client applications.
      • “QlikView” is more complex than “Qlik Sense”, i.e. for domain experts with good software programming skills.
      • Book Barry Harmsen, Miguel Garcia "QlikView 11 for Developers".
        • Accompanying website QlikView 11 for Developers.
      • Application developers need both the “QlikView” application server and rich client.
      • Important commands:
        • By “File / Edit Script” you may create and modify the query code to access and filter the data of the project.
        • By “File / Table Viewer” you may view the table structure of the project.
      • With “QlikView”, you must import each single spreadsheet worksheet, e.g if a spreadsheet file consists of 3 worksheets, you must do 3 manual imports.
    • “Qlik Sense”.
      • The free QlikTech International AB "Download Qlik Sense Desktop. Experience the power of Qlik on your personal computer", QlikTech International AB "Download von Qlik Sense Desktop. Kostenfreie Self-Service Datenvisualisierung und Discovery - vertrauen Sie Ihrer Intuition" for Win32 or Win64 - “Qlik Sense Desktop ist eine kostenfreie Windows Desktop Applikation, mit der Einzelpersonen über Drag-and-Drop interaktive Datenvisualierungen, Reports und Dashboards aus verschiedensten Datenquellen erstellen und teilen können. Gewinnen Sie in nur 5 Minuten mehr Einsichten in Ihre Daten!”.
      • “Qlik Sense” is a local server application for Win64 computers. You may operate with the desktop client “Qlik Sense Desktop” - which is a modified browser - or may use a standard Internet browser instead, by calling URLs:
        • QlikSense HTML Client "http://localhost:4848/hub".
        • QlikSense Developer HTML Client "http://localhost:4848/dev-hub".
      • Application developers just need the “Qlik Sense” (enterprise) application server, on which the application is published.
      • “Qlik Sense” is more simple than “QlikView”, i.e. for domain experts without software programming skills.
      • I was told by sales staff in 2016, that new customers prefer to order “Qlik Sense” prior to “QlikView”.
    • The online service QlikTech International AB "Join Qlik Sense Cloud. Experience the power of Qlik on your personal computer" with free entry-level account offer.
      • With “Qlik Sense”, you may import a spreadsheet at once, e.g if a spreadsheet file consists of 3 worksheets, you must just do one import.
  • Experts told me about filtering:
    • Filters work as associative logic in context.
    • A number of filters may may be applied to an Qlik application.
    • If you apply data filtering to the Qlik data model, it is applied to all Qlik applications which use it.
    • If you apply data filtering to a Qlik application, it is applied to the single Qlik application.
  • Design rules for dashboards:
    • Simple rule:
      • Left side = “Filter”.
      • Right side = “Information”.
    • Resources:
      • Google - Search for "qlik design rules for dashboards".
      • Qlik Community "Guide to Dashboard Design.pdf" 2014 → PDF.
      • Qlik Community "design guidelines for qlikview", 2013.
      • Qlik Community "Design Guidelines", 2012.
• The commercial Java application Pentaho Corporation, Pentaho Corporation - Germany.
  • it-novum GmbH - German service partner.
  • Pentaho Community.
    • The free Pentaho BI Platform / Server. The Pentaho BI platform provides architecture and infrastructure required to build solutions to business intelligence (BI) problems = “Pentaho BI Platform Community Edition (CE)”.
    • The OpenSource Pentaho Reporting Project. A collection of primarily focused on the creation, generation and distribution of rich and sophisticated reporting from all sources of information = “Pentaho Reporting Community Edition (CE)”.
    • The OpenSource Pentaho Kettle Project. A powerful, metadata-driven ETL tool designed to bridge the gap between business and IT; Turning company's data into increased profits = “Pentaho Data Integration Community Edition” ( PDI CE ).
    • The OpenSource Penatho Modrian Project. Open source analysis OLAP server written in Java. Enabling interactive analysis of very large datasets stored in SQL databases without writing SQL - “Pentaho Analysis Services Community Edition”.
    • The OpenSource Pentaho Weka Project. A comprehensive set of tools for machine learning and data mining used to help you understand the business better through predictive analysis - “Pentaho Data Mining Community Edition (CE)”.
  • The OpenSource Java application SourceForge "Pentaho Business Analytics", SourceForge "Pentaho Business Analytics".
  • EN.Wikipedia "Pentaho", DE.Wikipedia "Pentaho" - “Pentaho Data Mining used the Waikato Environment for Knowledge Analysis (Weka) to search data for patterns”.
  • Twitter "Pentaho".
  • Third-party applications.
    • Google Code " pdi-twitter. Using Simple Twitter API with Pentaho Data Integration (PDI) version 4.2".
• The commercial SAS software.
  • The free SAS University Edition - “Kostenlose Statistik Software”.
• The OpenSource commandline application VisiData - “An interactive multitool for tabular data. It combines the clarity of a spreadsheet, the efficiency of the terminal, and the power of Python, into a lightweight utility which can handle millions of rows with ease”.
  • GitHub "saulpw/visidata".
• The OpenSource application Weka 3: Data Mining Software in Java.
  • EN.Wikipedia "Weka (machine learning)", DE.Wikipedia "Waikato Environment for Knowledge Analysis".
• The announced “XDEV BI Suite” of “XDEV 4” ?!
  • XDEV Software Corp..

• The OpenSource framwork CERN "ROOT".
  • EN.Wikipedia "ROOT", DE.Wikipedia "ROOT".
  • ROOT software script can also be executed by the C++-Interpreter CINT.
• The OpenSource Python library “pandas” → See Python.

• The OpenSource OpenRefine, GitHub "OpenRefine" ( formerly: “Google Refine” ) - “A powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data”.
• EN.Wikipedia "OpenRefine".

• IBM Watson
• BigML
• SAS Enterprise Miner
• R platform
• Python machine learning libraries
• Google Prediction API
• WEKA Machine Learning Workbench
• Microsoft Azure Machine Learning
• AWS Machine Learning
• TensorFlow
• PredictionIO
• MATLAB or Octave
• DMTK
• IBM SPSS Modeler
• Apache Spark
• RapidMiner

• The OpenSource library Tensorflow, GitHub "tensorflow" - “TensorFlow is an Open Source Software Library for Machine Intelligence”, “Open source software library for numerical computation using data flow graphs”.
• GitHub "google/skflow" - “Simplified interface for TensorFlow (mimicking Scikit Learn)”.

• BGR "Google wants everyone to have access to its machine learning technology", 2015-11-09.
• KDnuggets - Data Mining, Analytics, Big Data, and Data Science "TensorFlow Disappoints – Google Deep Learning falls shallow", 2015-11.
• Lifehacker Australia "Google's TensorFlow Could Be A Game-Changer In The Future Of AI", 2015-11.
• Tech Insider "Google's new artificial intelligence tool has massive potential, 2015-11-10.
• Wired "Google Open-Sourcing TensorFlow Shows AI’s Future Is Data", 2015-11-16.

• Microsoft Power BI, Microsoft Power BI.
  • The free dashboard tool “Power BI Desktop for Windows” for Win7, Win8 - “Analytics tools at your fingertips”.
  • The publishing service “Power BI”, just for users of the commercial “Office 365” service?! - “The easy way to see your important data in one place. With a few clicks, connect to data from applications you use and get started with pre-built dashboards from experts”.
• Microsoft Power BI Community.
  • Blog Microsoft Community Blog.
  • Microsoft Power BI "Power BI Support. Get help and support for Power BI".
  • Microsoft Power BI Community "I fail with simple two-row data visualisation - please help".

• Microsoft Download Center "Microsoft Power BI Desktop".
  • Just for users of the commercial “Office 365” service?!
• Windows Store "Microsoft Power BI".
• Google Play "Microsoft Power BI" for Android smartphones, “Requires Android 4.4 and up”.
  • Just for users of the commercial “Office 365” service?!
  • But Google Play refuses installation “Your Device isn´t compatible with this version” on simulator Bluestacks with “Android 4.4.2”.

• “Microsoft Power BI” is a dashboard tool, not a reporting tool!
  • It is powered by “Microsoft SQL Server Analysis Services (SSAS)”.
  • It is now easy to catch data from HTML pages ( e.g. data tables of Wikipedia ).
  • Data visualisation:
    • Built-in graphics ( by a graphics engine similar / identical to that known from Micrsoft Excel, Microsoft SQL Server Reporting Services and Microsoft SQL Server Analysis Services ).
    • There is an API by which third parties may offer own visualisations.
  • Indeed with Microsoft Excel 2016, you may create the same dashboards, but you can´t publish it on the web.
• You may save “Microsoft Power BI Desktop” as in single ”.pbix” files. Experts told me, that for saving a project, the software needs 3 times of the RAM memory as needed for the loaded data. So it might be that you may load big data, but are not able to save it. In general, this might not be a problem with the 32-bit edition ( on Windows PCs with 2 or 3 GByte of RAM ), but also with the 64-bit edition ( i.e. a project may be saved on a Win64 computer with 16 GB RAM, but not on a Win64 computer with 4 GB RAM ).
• You might create test cases ( e.g. at business intelligence trainings ) with “Microsoft Power BI”, as alternative to NBI.
  • NBi, GitHub "Seddryck/NBi", CodePlex "NBI" - “Testing framework for Microsoft Business Intelligence platform”.
• “Microsoft Power BI Desktop” also supports the “M” and “DAX” query languages, by the “Advanced Editor” ( “Home / Edit Queries ( = Query Editor ) / Advanced Editor” ).
• If you install “Microsoft R” prior to “Microsoft Power BI Desktop”, you may use “R” for data processing and graphical reporting → See Mathematical Engineering.
• I was told by experts, in 2016-03:
  • Now legacy versions of “Microsoft Power BI Desktop” were able to process 10.000 datasets for visualisation.
  • Current versions of “Microsoft Power BI Desktop” were able to process 10.0000 datasets for visualisation.
  • With “R” called by “Microsoft Power BI Desktop”, you may visualize 150.000 datasets.

• See also Databases, Webserver, Webservices, SQL, XML.

• Microsoft Download Center "Power BI Gateway – Personal".

• Microsoft Power BI "Power BI Documentation" - "Power BI Gateway - Personal".
• Microsoft Power BI "Power BI Documentation" - "Troubleshooting Power BI Gateway - Personal".

• To get a BI report, connect with a client ( Internet browser, smartphone app,..) to the “Microsoft Power BI” cloudservice. The basic “Power BI” account is free.
• By this, you may access any cloud document.
• The cloud servce may access a registered onpremise Windows server at your ( the datacenter of your ) company, if the computer runs a free “Power BI Gateway - Personal” service, by a VPN connection.
  • Experts told me, that the free VPN service shipped with Windows is not under development for some time. Therefore it isn´t very advisable to use it in production environments.

• Business Intelligence Insight "Power BI Personal Gateway, Five Things You Must Know".
• RADACAD Blog "Definitive Guide to Power BI Personal Gateway".
• #sysctr.info "Installing the Power BI Personal Gateway as a Service Account - Revisited".

• Microsoft PowerPivot.
• Microsoft Office 2013 - Business Intelligence.
  • Powerful BI Now Available in Excel 2013:
    1. xVelocity In-memory Analytics.
    2. Quick Explore, Quick Analysis and Flash Fill.
    3. Power View.
    4. Power Pivot.
• Microsoft Power Query for Excel ( formerly: Microsoft Data Explorer for Excel ).
  • Microsoft Power BI for Office. Self-service Business Intelligence for all your data.
  • Microsoft Download Center "Microsoft Power Query for Excel".
  • Microsoft Power Query for Excel Help.
  • Blogs Microsoft MSDN "Business Intelligence Blogs"
    • Blog Microsoft MSDN "Power BI" ( formerly: Microsoft MSDN Blog “Data Explorer Microsoft 'Data Explorer' Preview for Excel” ).

• Microsoft MSDN Developer Network "Query and Data Modeling Languages".

• See on this page.

• Microsoft Docs "Analysis Services Scripting Language (ASSL for XMLA)".
• Microsoft Docs "XML for Analysis (XMLA) Reference".
• Microsoft TechNet "Introducing Analysis Services Scripting Language".
• Microsoft TechNet "XML for Analysis (XMLA)".
• EN.Wikipedia "XML for Analysis" ( XMLA ).

• Microsoft MSDN Developer Network "DAX Syntax Reference".
• Tbe OpenSource software SQLBI "DAX Studio", CodePlex "DAX Studio" for Windows.
  • Excelerator BI "Getting Started with DAX Studio".
  • Experts told me, that you may connect from “DAX Studio” to a (running) PowerBI model.

• Microsoft MSDN "The Microsoft code name "M" Modeling Language Specification - Introduction".
• pqreference.azurewebsites.net.
  • PowerQuery Formula Reference, August 2015 ( PDF ).
  • PowerQuery Formula Language Specification, August2015 ( PDF ).
• Blog Microsoft MSDN Blog "Codename 'M' Language Team Blog".
• Blog Microsoft MSDN Blog "Model Citizen".
  • Microsoft MSDN Blog "Model Citizen: Update on SQL Server Modeling CTP (Repository/Modeling Services, 'Quadrant' and 'M')".
• InfoQ "Codename 'M': Language, Data, and Modeling, Oh My!".
• EN.Wikipedia "M (programming language)", EN.Wikipedia "Oslo (Microsoft)".

• Tbe OpenSource software ]SQLBI "MDX Studio"], [[http://daxstudio.codeplex.com/|CodePlex "MDX Studio v2" for Windows.
• Microsoft.
  • Microsoft Docs "Multidimensional Model Data Access (Analysis Services - Multidimensional Data)".
  • Microsoft Docs "Multidimensional Expressions (MDX) Reference".
• EN.Wikipedia "MultiDimensional eXpressions", DE.Wikipedia "Multidimensional Expressions".

• Experts told me, that “Tabular” is like “PowerBI” / “QlickView” / “Tableau” but without graphical user interface and graphical reporting. Instead, “Tabular” may be accessed by any tool by DAX & MDX query language.
• Microsoft Docs "What is Analysis Services?".
  • Microsoft Docs "Tabular Model Programming for Compatibility Level 1200 and higher".
  • Microsoft Docs "Tabular Model Scripting Language (TMSL) Reference".
• SQLBI "Adding a measure to a Tabular model".

• Datazen - Mobile BI and Data Analytics for Any Device.
  • “Microsoft Acquires Datazen”, 2015-04-14.
  • The functionality of Datazen is included in “SSRS 2016” for
    1. Windows 10 Desktop ( but not Windows Phone! ).
    2. Android.
    3. iPhone.

• The commercial online course Beyond Jupyter - Take your data science skills to the next level.
  • Beyond Jupyter "Legal Disclosure" with E-Mail address of “Joshua Görner”.
• Autor XING "Joshua Görner", LinkedIn "Joshua Görner", GitHub "Joshua Görner".
  • GitHub "jgoerner/data-science-stack-cookiecutter".
• Free conference SIGS DATACOM GmbH "Code Days" as part of the conference OOP 2018 at the ICM - Internationales Congress Center München, München ( 2018-02-17 - 2018-02-18 ).
  • SIGS DATACOM GmbH "Code Days" - "Programm-Details", 2018-02-18 - “Joshua Görner: Beyond Jupyter Notebooks: Build your own Data Science Platform in less than 1 hour”.
• On the TDWI 2018 conference in München, Germany, Joshua Görner ( TDWI Konferenz 2018, Sprecherdetails "Joshua Görner" ) was speaker. I joined his seminar “Beyond Jupyter Notebooks: Build yoru own Data Science Platform in less than 1 hour” .
• Suggested “Data Science Platform”:
  1. Jupyther.
     • See Python 1/10.
  2. Superset.
     • Apache Superset.
  3. PostgreSQL / PostgREST.
     • See Data Processing 1/5 - Databases, Webserver, Webservices, SQL, XML.
  4. minio.
     • Minio - “Private Cloud Storage. Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure. Minio is widely deployed across the world with over 105.1M+ docker pulls”.
  5. API Star.
     • API Star, GitHub "encode/apistar" - “A smart Web API framework, for Python 3”.
     • API Star "Docs".
  6. Docker.
     • See Virtual PCs, Virtual Servers, Virtualization, Virtual Computers, Computer Hardware Simulation.
     • Book Joshua Cook "Docker for Data Science: Building Scalable and Extensible Data Infrastructure Around the Jupyter Notebook Server".
• Joshua Görner suggested to use the Python web framework “Flask” to build datascience applications. “Flask” is providing easily “Functions as a Service” for datascience applications. See Python 10/10 - Web Application Frameworks.

• GitHub "ThoughtWorksInc/twde-datalab" - “Onboarding to data science by ThoughtWorks”.
• The speech “Mi 9.1: A Hands-on Introduction to Your First Data Science Project” by A. Wider, E. Grasmeder at the conference TDWI Konferenz 2018 by TDWI Germany e.V., 2018-06-25 - 2018-06-27, introduced me to the “TWDE Datalab” .

• Bissantz & Company GmbH "DeltaMaster: Software für Analyse, Planung und Reporting".
• Informatica - The Data Integration Company.
• INTEC International GmbH - “Monitoring von Geschäftsdaten mit dem BigData Monitor”.
  • Hannover Messe "BigData Monitor".
  • The graphical data organisation scheme is:
    1. Size of a rectangle.
    2. 3 colors ( transition from red to white and then to blue ).
    3. Label for each rectangle.
  • This is an adoption of the product idea of Technische Universität Eindhoven "SequoiaView".
• prudsys - The Realtime Analytics Company.
  • prudsys - The Realtime Analytics Company "Clever Empfehlen".
  • Facebook "prudsys.de".
• TIBCO Software Inc..
  • The commercial software TIBCO Software Inc. "Spotfire".
    • OnTheHub "Download Spotfire S+ 8.0 for Windows Student Edition - Free “TIBCO Spotfire S+ 8.0 (1 Year License)”.
  • TIBCO Developer Network.
  • it-production.com -Das Online-Magazin für erfolgreiche Produktion "Zwischen Top Floor und Shop Floor 'Die Zwei-Sekunden-Lücke nutzen'" - “Die Frage, ob sich Business Intelligence-Systeme für Auswertungen im Produktionsumfeld genauso eignen wie für Forecast-Aufgaben für das Management..”.
  • DE.Wikipedia "TIBCO Spotfire Analytics".

• The online cloud based predictive analysis service BigML - “Machine Learning for everyone. Easily add data-driven decisions and predictive power to your company”, “NOW FREE. Unlimited tasks ( up to 16 MB/task )”.

• EdgeImpulse Inc..
  • EdgeImpulse Inc. "Pricing" - Free developer account ( “Developer. For developers looking to deploy innovative ML on any edge device = Free” ).
  • EdgeImpulse Inc. Docs.
    • EdgeImpulse Inc. Docs "Development Boards" - “There is a list of development boards that are fully supported by Edge Impulse”.
• Onsite event EdgeImpulse Inc. "Presenting Edge ML Series. Frankfurt, Amsterdam, Milan, Paris, London, Munich".
  • 27-28 April Frankfurt - (27.4.2022 12:00-18:00 and 28.4.2022 8:30-13:00).
  • 3 May Amsterdam - (3.5.2022 all day), 08:30-18:00 ( 17:00 ) ⇒ I applied and was accepted .
    • Renaissance Amsterdam Hotel, Kattengat 1, 1012 SZ Amsterdam, Netherlands.
    • Smartphone app:
      • Marriott "Marriott Bonvoy App".
      • Google Play, Marriott International "Marriott Bonvoy".
    • Deutsche Welle "Einreise- und Corona-Regeln: Niederlande", 2022-04-20 - “Einreiseregeln. Die Niederlande gilt aus deutscher Sicht nicht mehr als Hochrisiko-Gebiet. Für Reiserückkehrer aus den Niederlanden entfällt damit die Digitale Einreiseanmeldung, für Ungeimpfte ist keine Quarantäne mehr nötig”, “Seit dem 23. März ist die Einreise in die Niederlande ohne Nachweispflicht möglich. Im Land wurden fast alle Corona-Regeln aufgehoben”.
  • 5-6 July Munich - (5.7.2022 12:00-18:00 and 6.7.2022 8:30-13:00).

• The free online service Google Colab ( Colaboratory ).
• A free Jupyter Notebook server, most packages of Anaconda are installed, but not 100% compatible ( different version of Python packages, few missing packages ).
• User experience of Google Drive users ( so you must be logged into Google Drive by your Google account ):
  • Create new Jupyter cells.
    • Button ”+Code” and menu option “Insert / Code cell” create a new code cell, while ”+Text” and menu option “Insert / Text cell” create a new text cell, at the point of the Jupyter notebook, where your cursor is, i.e. where you have hit your mouse by mouse-leftlick.
    • In local Jupyter notebooks, there is a button of the extra button bar to provide this. This button bar is not available with Google Colab.
  • You may create a copy of a Jupyter notebook put on Google Docs, by “File / Save a copy in Drive”.
  • But its most easy to transfer additional data files by downloading them on your local terminal ( PC with browser ), and to upload it into the copied Jupyter Notebook.
  • You might use “Files / Mount Drive” to create the code

    from google.colab import drive
    drive.mount('/content/drive')
    

• Microsoft "Azure Kusto" - “Kusto is a log analytics cloud platform optimized for ad-hoc big data queries”.
• Microsoft KUSTO "Documentation", Microsoft KUSTO "Documentation, Microsoft KUSTO "Documentation - Not accessable with simple free Microsoft account .
• Microsoft Docs "Azure Kusto (Preview)".
• GitHub "Microsoft Azure Kusto Library for Python".

• Azure Log Analytics Query Language.

• SquaredUp "Kusto: A new query language for OMS Log Analytics".

• The online cloud based service SAP HANA, with free developer account.
  • SAP HANA Cloud Platform - Login for the free developer account.
    • SAP HANA Cloud Platform "Cockpit".
  • SAP HANA Cloud Platform Help.
    • SAP HANA Cloud Platform Help "Signing Up for a Developer Account" - “The name of your new developer account contains your user ID ( e.g. 'p0123456789' ) and the suffix trial ( e.g. 'p0123456789trial' )”. However, you still have to login with your SAP ID or E-Mail address, not your developer ID.
  • SAP Community Network.
    • SAP Developer Center, SAP Community Network Developer Center.
      • SAP Community Network "Smart Data Streaming".
  • SAP HANA Developer Center - Signup here for a free developer edition account of SAP HANA in the cloud .
  • SAP Cloud Appliance Library.
• As of 2016-02, you may install developer editions on SAP HANA Cloud Platform, Microsoft Azure Cloud, Amazon Web Services only. Experts expect, that after 2016-05 there will be a downloadable developer edition and / or a entry-level commercial edition of SAP HANA.

• SAP HANA Cloud Platform "Tutorial Catalog".
• Webcasts.
  • openSAP "Introduction to SAP HANA Cloud Platform (Repeat)".
  • openSAP "Next Steps in SAP HANA Cloud Platform".
  • openSAP "How the Internet of Things and Smart Services Will Change Society".
• YouTube Channel "SAP HANA Cloud Platform".
  • YouTube Playlist "SAP HANA Cloud Platform".

• SAP Codejam.
  • SAP Community Network "Hosting an SAP CodeJam Event".
  • Software to install for the ”SAP Codejam (SAP HANA with IoT)”:
    • Node.js.
      • ”Node.js 0.12.10”, for use with “Tessel1” boards. See → ARM Embedded Systems ( ARM7, ARM8, ARM9, ARM11, Cortex ).
      • NodeJS for use with Texas Instruments Sensortag. See → Wireless Networks.
    • Browser Google Chrome.
    • Postman, Google Chrome Webstore "Postman".
    • HTML/JavaScript editor.
  • SAP Community Network, SAP CodeJam "Recap IoT CodeJam Hannover at inwerken AG".

• EN.Wikpedia "SAP HANA", DE.Wikpedia "SAP HANA".

• Justin Langseth "Real-Time Data Warehousing: Challenges and Solutions".

• See FIWARE 1/2.

• Google Cloud "Stream analytics solution".

• Microsoft Azure "Azure Stream Analytics".
• Microsoft Docs "Stream Analytics Documentation".
  • Microsoft Docs "Stream Analytics Documentation" - "Stream data as input into Stream Analytics".
    • “Azure Event Hubs provides highly scalable publish-subscribe event ingestors. An event hub can collect millions of events per second, so that you can process and analyze the massive amounts of data produced by your connected devices and applications”.
    • “Together, Event Hubs and Stream Analytics provide an end-to-end solution for real-time analytics. Event Hubs lets you feed events into Azure in real-time, and Stream Analytics jobs can process those events in real-time”.
      • “For example, you can send web clicks, sensor readings, or online log events to Event Hubs. You can then create Stream Analytics jobs to use Event Hubs as the input data streams for real-time filtering, aggregating, and correlation”.
  • Microsoft Docs "Stream Analytics Documentation" - "Get started with Azure Stream Analytics to process data from IoT devices".
    • “In a real-world scenario, you could have hundreds of these sensors generating events as a stream. Ideally, a gateway device would run code to push these events to Azure Event Hubs or Azure IoT Hubs. Your Stream Analytics job would ingest these events from Event Hubs and run real-time analytics queries against the streams. Then, you could send the results to one of the supported outputs”
• GitHub "Microsoft Azure Stream Analytics" - “This repository contains data samples and data generators used for Azure Stream Analytics tutorials”.

• Microsoft Library "Microsoft StreamInsight".
• Microsoft Library "Columnstore Indexes" - Additional technology for “Realtime Data Warehouses”.

• See on this page.

• “Data Ingest” ( German: “etwas ( Nahrung ) zu sich nehmen”, “Daten einfügen” ) - If you process data - by machine learning, by statistical methods -, the resulting data is added ( “ingested” ) to the original data. The term “insert” is not used, as it is already used with SQL.
  • Google - Search for "Data Ingest".
• Dataversity "Streaming Analytics 101: The What, Why, and How".
  • “SQLstream: Blaze enables companies to be data-driven in real time. Real-time data can be discovered, analyzed, and aggregated instantly, and delivered as a continuous ingest into Hadoop, data warehouses, and other enterprise systems”.
• EN.Wikipedia "Complex event processing".
• DE.Wikipedia "Edge computing", DE.Wikipedia "Edge Computing".
• EN.Wikipedia "Azure Stream Analytics".

• Grafana.
  • See Diagrams & Graph Visualization.
• Kibana.
  • The OpenSource plugin Elasticsearch Kibana - “Explore & Visualize Your Data”.
  • Elastic "Learn, Docs" - "Kibana".
    • Elastic "Learn, Docs" - "Kibana Dashboard" - “A Kibana dashboard displays a set of saved visualizations in groups that you can arrange freely. You can save a dashboard to share or reload at a later time”.
  • Kibana 4 Tutorial - “A web frontend to analyze data held in an elasticsearch cluster”.
  • EN.Wikipedia "Kibana" - “Open source data visualization plugin for Elasticsearch”.
  • EN.Wikipedia "Elasticsearch", DE.Wikipedia "Elasticsearch".

• Elitness.
  • Google - Search for "elitness bm25".
  • Google - Search for "The Probabilistic Relevance Framework BM25 and Beyond".
  • EN.Wikipedia "Okapi BM25".
  • ACM Digital Library "The Probabilistic Relevance Framework: BM25 and Beyond".
  • Stephen Robertson - Papers.
    • The Probabilistic Relevance Framework: BM25 and Beyond ( PDF ).
      • “3.4 The Eliteness Model and BM25. We now re-introduce term frequencies into our model... We suppose that for any document-term pair, there is a hidden property which we refer to as eliteness. This can be interpreted as a form of aboutness: if the term is elite in the document, in some sense the document is about the concept denoted by the term. Now we assume that actual occurrences of the term in the document depend on eliteness, and that there may be an association between eliteness (to the term) and relevance (to the query)”.
    • EN.Wikipedia "Elasticsearch", DE.Wikipedia "Elasticsearch".
• EN.Wikipedia "Full text search", DE.Wikipedia "Volltextrecherche" - “Zur schnellen Informationsgewinnung und dem Auffinden aus bekannten wie auch nicht bekannten (aber auf den Medien vorhandenen) Dokumenten wird die Volltextrecherche genutzt. Die Volltextrecherche dient daher dem Auffinden, Entdecken und Extrahieren unbekannter, nicht trivialer und wichtiger Informationen aus großen Mengen von unstrukturierten Texten/Dateien und ist somit auch ein wichtiger Teilbereich des Text Mining. Sie ist eine Sofortlösung, um für eine konkrete Fragestellung auf Systeme wie Dokumentenmanagement und Data-Mining verzichten zu können”.
  • Document management → See Content Management Systems ( CMS ).
  • Data-Mining → See “Business Intelligence” on this page.

• AI Spektrum: Ihr Fachportal für Künstliche Intelligenz.

• evidanza AG - BI-Webcast.
• Compuware "On-Demand Webcasts".
  • Webcast “Essentials of Business Analytics & Big Data: What Every Service Leader Needs to Know”.

• AandBI.com powered by Grete DV GmbH ( XING "Markus Alexander Grete" ), Burgdorf - “Implementierung und Schulung KI, BI, Datenschutz, Implementation and Education, Artificial Intelligence, Business Intelligence and Privacy Law”.

• Book Viktor Mayer-Schonberger, Thomas Ramge: "Reinventing Capitalism in the Age of Big Data", 2018.
• PDF book TDWI "The Essential Guide to Machine Data", 2019-07.
• PDF book MemSQL, Aileen Nielsen "Cutting Edge Approaches to Time Series Analysis", 2019-08.

• While commercial editions of OpenSource BI software tools start with 10.000 EUR/a... 50.000 EUR/a for single customers, there are totally different costs, if the customer wants to integrate or bundle the commercial edition of the OpenSource BI software tool with the own software solution to end-customers, as OEM partner .
• I was told by experts, that some business intelligence software tools are not based on multidimensional databases, but SQL, NoSQL,...
• The competition Data Mining Cup.
• Data Science Central - “The Online Recourse for Big Data Practitioners”.
• enterprise application world @ Hannnover Fair CEBIT.
• Kaggle: Your Home for Data Science.
• KDNuggets - “Data Mining Community's Top Resource”.
• PASS - The Professional Association for SQL Server.
• QlikCommunity, Wiki.
• Speicherguide "Big Data heißt: Loslassen von strukturierten Daten" - “Polystrukturierte Daten ist ein neuer Begriff, der strukturierte und unstrukturierte Daten sowie die neuerdings massiv auftretenden maschinengenerierten Daten wie Sensor-Daten und Web-Logs mit einbezieht”.
• Blogs THE SQL Server Blog Spot on the Web.
  • Blog THE SQL Server Blog Spot on the Web "SQLBI - Marco Russo" - “SQLBI is a blog dedicated to building Business Intelligence solutions with SQL Server”.
• Wikipedia.
  • EN.Wikipedia "Cross Industry Standard Process for Data Mining" ( CRISP-DM ).
  • EN.Wikipedia "Data Mining", DE.Wikipedia "Data-Mining".
  • EN.Wikipedia "Star schema", DE.Wikipedia "Sternschema".
  • EN.Wikipedia "Data integration", DE.Wikipedia "Informationsintegration".
  • EN.Wikipedia "Operational data store", DE.Wikipedia "Operational Data Store".
  • EN.Wikipedia "Data stream mining".

• ApatarForge Forums.
• BIRT Exchange Forums.
• CloverETL Forum.
• ENZEE - The Netezza Community.
• Facebook.
  • Facebook "Pentaho", Pentaho Business Intelligence - Joaçaba. Community Page about Business Intelligence Consultant.
  • Facebook "SAP Developers".
• Informatica Commmunities.
  • Informatica User Groups.
    • Informatica User Groups - Discussions.
• JasperForge.
• Kaggle: Your Home for Data Science "Discussion".
• LinkedIn:
  • LinkedIn "Advanced Business Analytics, Data Mining and Predictive Modeling".
  • LinkedIn "Business Intelligence Elite".
  • LinkedIn "MicroStrategy Business Intelligence".
• Microsoft
  • Microsoft Power BI Community "Power BI User Groups".
    • Microsoft Power BI Community "Power BI User Groups", "Power BI User Group Hamburg".
  • Microsoft Power BI Community "Forums".
  • Microsoft Technet Forums - Questions tagged with "powerbi" - “All Power BI related forums have been moved to http://community.powerbi.com/]. These forums are now locked and read-only. We sincerely apologize for the inconvenience this may have caused you”, 2015-07-17:
    • “Power BI Dashboard Preview”.
    • “Power BI for Office 365”.
    • “Power BI Mobile Apps”.
    • “Power BI Q&A”.
• MicroStrategy Discussion Forums.
• Pentaho Community Forums.
• Qlik Community.
  • QlikView Forums & Resources.
  • Qlik Sense Forums.
• SAP Community Network Forums.
• SAP Wiki.
• TalendForge - Talend community forum.
• TIBCO.
  • TIBCOmmunity.
  • The read-only ancient Forum TIBCO Spotfire Community.
  • TIBCOmmunity Analytics.
• Varigence Forums.