Trace: » databases.html
Table of Contents

is

[hemmerling] Data Processing 1/7 - Databases, Webserver, Webservices, SQL, XML

Organizations, Events

Organizations

Events

Free Databases, ETL Tools and Data Integration Tools

Apache Spark

BlackRay Data Engine

CrateDB

  • The OpenSource CrateDB - “Real-time SQL Database for Machine Data & IoT”.
  • EN.Wikipedia "CrateDB" - “CrateDB's language is SQL but it uses the document-oriented approach of NoSQL style databases”.

Ingres, PostgreSQL / postgres

The Tools

  • The OpenSource SQL database Ingres Corporation "Ingres".
  • “postgresql-8.4.13-1-windows.exe” and “postgresql-9.0.8-1-windows.exe” are the latest editions for W2k ( Download: OldApps "PostgreSQL" ).
  • The installation of “postgresql-9.0.8-1-windows.exe” on W2k causes a warning at the end of installation “A non-fatal error occured during cluster initalisation. Please check the installation log in C:\Temp for details”.
  • Installation of Postgres 9.4:
    • Password for the database superuser (postgres) → e.g “admin”.
    • Port number → default port number is “5432”.
    • Locale to be used by the new database cluster → Locale ”[Default locale]”.

Webserver with REST Interface, with PostgreSQL Server as Backend

  • PostgREST - “A standalone web server that turns your PostgreSQL database directly into a RESTful API. The structural constraints and permissions in the database determine the API endpoints and operations.”.

Tips&Tricks

  • Configuration suggestions:
    • Standard user: “postgres”. The password may be set during installation.
    • Standard SQL server port: “5432”.
    • For users in Germany:
      • Select the locale to be used by the new database cluster. Local “English, United Kingdom”. Do not set it to ”[Default locale]”.
      • If you use a Windows computer with local Windows settings for Germany ( e.g. ”,” as decimal mark ),
        • no matter if you use an English or German operating system,
        • and if you set the database locale to ”[Default locale”],
        • You will get all database messages in German language :-(.

Resources

Interbase / Firebird

Java Databases

Apache Derby ( formerly: IBM Cloudscape )

db40

HyperSQL ( HSQLDB ) / HypersonicSQL

MaxDB / SAP DB

MySQL Family

Oracle MySQL

MariaDB

Scalable Solutions for MySQL & MariaDB

Third-Party Scalable Solutions

Third-party Tools

  • The OpenSource backup software MySQLDumper for MySQL, written in PHP und Perl.

SQL Language

Resources

SQLite

NoSQL Databases

SQL Abstraction Applications

NoSQL Databases with SQL interface

    • “Crate.IO has built a new breed of database to serve today’s mammoth data needs. Based on the familiar SQL syntax, Crate combines high availability, resiliency, and scalability”.
    • “Why choose Crate Data? You can build similar solutions by combining a medley of technologies, like Mongo, Elasticsearch, Hadoop, CouchDB, Riak, Redis, MySQL, Postgres ect.”.
    • “Crate is a data store with the benefits of a NoSQL document with powerful search ( Elasticsearch as library included ), accessable using SQL.
    • I met CRATE representative XING "Stephanie Hollenstein" on the Europython 2014 fair.

SQL Query Engine

Free and Commercial Databases

Exasol

IBM

IBM DB2

  • IBM DB2 Express-C - “Free to develop, free to deploy, free to distribute”.
  • DB2 Express Edition - “Easy to Use. Low Cost. Powerful”.
  • EN.Wikipedia "DB2" - “DB2 Express-C is also similar to the free versions of Oracle database and Microsoft SQL Server, except that DB2 Express-C has no limit on number of users or on database size”.
    • Limitations of IBM DB2 Express-C:
      • “Benutzung von max. 2 Kernen einer CPU (bzw. 4 Kerne mit zusätzlichem Wartungsvertrag)”.
      • “Benutzung von max. 2 GB Hauptspeicher (bzw. 4 GB mit zusätzlichem Wartungsvertrag)”.
      • “Diese Version hat keine Einschränkungen hinsichtlich der Größe der Datenbank und der Anzahl der Benutzer, ohne zusätzlichen Wartungsvertrag gibt es jedoch keine Replikation, 24/7-Support und komfortable Updates”.

IBM Informix

Microsoft SQL Server

Microsoft

Microsoft SQL Server 2000 Desktop Engine (MSDE 2000)

Microsoft SQL Server 2005 SP4

Microsoft SQL Server 2008 R2 SP2

Microsoft SQL Server 2012

Microsoft SQL Server 2014

Microsoft SQL Server 2016

Microsoft SQL Server 2017

Microsoft SQL Server vNext

Additional Tools

Integration Services / Data Transformation Services

The Service
Automatic Generation of SSIS Packages
Resources
  • The motivation for “Integration Services” and SSIS:
    • Graphical design process “by mouseclick”, to determine and to connect and configure data sources for data transfer to and from an Microsoft SQL Server.
    • Internally, the Integration Services use components ( DLLs ) of Micosoft SQL Server, e.g. for lookup and fuzzy logic.
    • The Integration Services generate SSIS packages, in XML-based DTSX format. SSIS packages and its XML code do not allow any flow control like for-loops and if-then-else like any programming language.
    • Its easy to generate a handful of SSIS packages by the graphical design tools of IS :-), but a nightmare if you must generate 1000 packages. The approach for IS automation is to generate the XML code by third-party tools, by an own script language which supports usual programing language structures like for-loops and if-then-else.

Foren:

Additional Tools

Experts & Books & Tools

Resources

Software Edition User Password Comment
SQL Server 2008/R2 Express sa Blank password - leave field empty to connect
SQL Server 2012 Express sa Password123
SQL Server 2008/R2/2012 Web or Standard sa ? Can't be blank. Will be the same as your administrator or root user password at the time the Windows system was provisioned

Oracle Database

The Tool

Oracle Application Express ( APEX )

The Framework
Services

Resources

SAP

SAP OpenSQL

SAP Sybase

Products
Resources

Vertica

The Tool

  • Vertica Community Edition (CE) Free Trial
    • Store and analyze up to 1 TB of data for free with no time limit. Install Vertica CE on-premises, as a VM, on Apache Hadoop, or in your choice of clouds (AWS, Azure, Google).
    • Install Vertica CE on up to 3 nodes
    • Store and analyze up to 1 TB of structured and semi-structured data
    • Use Vertica for SQL on Apache Hadoop for data exploration as part of Vertica CE free trial
    • Enjoy no time limits or license requirements

Resources

Free and commercial integrated Data Management Applications

Free integrated Data Management Applications

Commercial Online Relational Database Services

Google Cloud Spanner

Microsoft Azure

The Service
Azure Simulator for Development
Azure & Java
Resources

Commercial integrated Data Management Applications

Commercial Tools for Databases & Database Managment

  • Oracle Data Integrator ( EN.Wikipedia "Sunopsis" → Sunopsis Data Conductor ).
    • microTOOL case/4/0 - “A tool for software analysis and design, data modeling and data warehousing. It supports modeling the functions, data and behavior of an application system through the use of function structures, information flows, state diagrams, ER models, data structures, data elements, relational models, module structures and type structures”.
    • The free limited “case/4/0 Personal Edition”.

Database Technologies

Database Drivers ( ODBC, OLE-DB, ADO-NET Data Provider... )

Tools

Resources

Database Design for Data Acquisition

  • Experts told me, that “In-Memory OLTP” is not suitable for data acquisition, as all the data ( tables ) must be stored in RAM, so that a rising data volume requires more RAM.
  • If a modern SQL server has enough RAM, the “only” permanent writing on the filesystem is the writing of SQL server logfiles ( “templog.ldf”) All other data is stored asynchronously.
  • Speed factors for writing database data:
    • HDD = 100 ( for comparison ).
    • SDD = 500.
    • FusionIO = 5000.

In-Memory OLTP

Memory-mapped Files

    • Microsoft Office does not make use of the “memory-mapped file” technology.
    • Windows Notepad makes use of the “memory-mapped file” technology.

Microsoft SQL Server

Multitenancy

Stored Procedures

Data Serialisation Formats

Apache Avro

JSON, JSONP, BSON

Protocol Buffers

XML

The Course "Datenmanagement mit SQL" by openHPI - Hasso Plattner Institut

The Course

Lecturers & Staff

Videos

Literature

General Books

General Papers

Papers about ER Modeling and relational Model

Books about ER Modeling and relational Model

Databases & Tools

Resources

Additional SQL Resources

The Course "Introduction to Databases" by Stanford University

The original Course of 2013

The original Course of 2011

Literature

Tools

XML Validation

XML Querying

XML Diff & Patch

XML, JSON Tools

Software
Free Online Services
Resources

SQL

Relational Algebra

Command Description
\help: print this message.
\quit: exit ra.
Ctrl-D
\list: list all relations in the database.
\sqlexec_{STATEMENT}: execute SQL in the database.
Relational algebra expressions:
R: relation named by R
\select_{COND} EXP: selection over an expression.
\project_{ATTR_LIST} EXP: projection of an expression.
EXP_1 \join EXP_2: natural join between two expressions.
EXP_1 \join_{COND} EXP_2: theta-join between two expressions.
EXP_1 \cross EXP_2: cross-product between two expressions.
EXP_1 \union EXP_2: union between two expressions.
EXP_1 \diff EXP_2: difference between two expressions.
EXP_1 \intersect EXP_2: intersection between two expressions.
\rename_{NEW_ATTR_NAME_LIST} EXP: rename all attributes of an expression.

Relational Theory

Resources

Using XPATH with Selenium 2 WebDriver

Tools for Firefox

Tools for Google Chrome

Resources

Literature

Books about SQL

Paper Magazines

Comparison of Databases / Product Selection

Services

Resources

  • Experts told me in 2022-06, about database performance, and database performance on hosts:
    • Dabases:
      • Couchbase & Cassandara can be scaled :-).
      • MongoDB, though so popular, is slow, and can´t be scaled :-(.
      • MariaDB / MySQL, though so popular, are slow and can´t be scaled too :-(.
    • Hosts:
      • IONOS/1&1 is not ok :-).

Resources

Communities, Forums, Newsgroups

Appropriate OpenDirectory Directory Pages

 
en/databases.html.txt · Last modified: 2024/03/17 07:21 (external edit) · []
Recent changes RSS feed Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki