Install Dependencies

1. Install Go
2. Install Mysql 5.6 or install Percona Server
3. Install the following other tools needed to build and run Vitess:

• make
• automake
• libtool
• memcached
• python-dev
• python-virtualenv
• python-mysqldb
• libssl-dev
• g++
• mercurial
• git
• pkg-config
• bison
• curl
• unzip

These can be installed with the following apt-get command

sudo apt-get install make automake libtool memcached python-dev python-virtualenv python-mysqldb libssl-dev g++ mercurial git pkg-config bison curl unzip

1. install JDK

Build Vitess

1. Set WORKSPACE environment variable

$WORKSPACE = /opt/vt/

1. Clone Vitess

cd $WORKSPACE
git clone https://github.com/youtube/vitess.git src/github.com/youtube/vitess
cd src/github.com/youtube/vitess

1. Set the MYSQL_FLAVOR environment variable.
   export MYSQL_FLAVOR=MySQL56
2. Run mysql_config –version and confirm that you are running the correct version of MariaDB or MySQL. The value should be 5.6.x for MySQL.
3. Build Vitess using the ./bootstrap.sh commands.
4. If error occurs with zookeeper download and extract archive, you can download it manually and copy it in this $VTROOT/dist path and comment bellow code in line 38 at bootstrap.sh

wget http://apache.cs.utah.edu/zookeeper/zookeeper-$zk_ver/zookeeper-$zk_ver.tar.gz && \

1. If install successfully done, you should see bellow message at at end of bottstrap.sh messages

example output:
skipping zookeeper build
go install golang.org/x/tools/cmd/cover …
Found MariaDB installation in …
skipping bson python build
creating git pre-commit hooks

source dev.env in your shell before building

1. If step 11 is unsuccessful and error is in install package with python pip we solve this problem in below steps

* Because gRPC installs new instance of python in its own path, we should install gprc, protobuf, zookeeper in /usr/ directory and for this, we should do some steps befor run new bootstrap version
+ Set password for root user

sudo passwd root

• Change user to root

su root

• Set /urs/ directory permission to 777

chmod 777 -R /usr/

• Change sudo file permision

a. chown root:root /usr/bin/sudo
b. chmod 4755 /usr/bin/sudo
c. mv -iv /etc/sudo.conf /etc/sudo.conf.old

• Copy and replace our new files into vitess directory and execute bootstrap.sh file again
• in this solution vitess use python of server and all library files will be install in /usr/ path

1. After run vitess successfully you should make vitess use this command

make build

1. If you encourage problem with go, in download from go.googlesource.com you should download flowing packages from these links and extract them in given path

1.
Package: net
Download: https://go.googlesource.com/net/+archive/master.tar.gz
Destination: &GOROOT/src/golang.org/x/net
2.
Package: oauth2
Download: https://go.googlesource.com/oauth2/+archive/master.tar.gz
Destination: &GOROOT/src/golang.org/x/oauth2
3.
Package: tools
Download: https://go.googlesource.com/tools/+archive/master.tar.gz
Destination: &GOROOT/src/golang.org/x/tools
4.
Package: gocloud
Download: https://code.googlesource.com/gocloud/+archive/master.tar.gz
Destination: &GOROOT/src/google.golang.org/cloud
5.
Package: google-api-go-client
Download: https://code.googlesource.com/google-api-go-client/+archive/master.tar.gz
Destination: &GOROOT/src/google.golang.org/api

Run Tests

1. After build vitess successfully, If you want only to check that Vitess is working in your environment, you can run a lighter set of tests

make site_test

Start a Vitess cluster

1. Set VTROOT to the parent of the Vitess source tree. For example, if you ran make build while in /opt/vt/src/github.com/youtube/vitess, then you should set:

export VTROOT=/opt/vt

1. Set VTDATAROOT to the directory where you want data files and logs to be stored. For example:

export VTDATAROOT = $VTROOT/vtdataroot

1. Servers in a Vitess cluster find each other by looking for dynamic configuration data stored in a distributed lock service. The following script creates a small ZooKeeper cluster:

cd $VTROOT/src/github.com/youtube/vitess/examples/local
vitess/examples/local ./zk-up.sh

example output :
Starting zk servers
Waiting for zk servers to be ready

After the ZooKeeper cluster is running, we only need to tell each Vitess process how to connect to ZooKeeper. Then, each process can find all of the other Vitess processes by coordinating via ZooKeeper.
Each of our scripts automatically sets the ZK_CLIENT_CONFIG environment variable to point to the zk-client-conf.json file, which contains the ZooKeeper server addresses for each cell.

1. The vtctld server provides a web interface that displays all of the coordination information stored in ZooKeeper.

vitess/examples/local ./vtctld-up.sh

Starting vtctld
Access vtctld web UI at http://localhost:15000
Send commands with: vtctlclient -server localhost:15999
Open http://localhost:15000 to verify that vtctld is running.

There won’t be any information there yet, but the menu should come up, which indicates that vtctld is running.

The vtctld server also accepts commands from the vtctlclient tool, which is used to administer the cluster. Note that the port for RPCs (in this case 15999) is different from the web UI port (15000). These ports can be configured with command-line flags, as demonstrated in vtctld-up.sh.

List available commands

$VTROOT/bin/vtctlclient -server localhost:15999 Help

1. The vttablet-up.sh script brings up three vttablets, and assigns them to a keyspace and shard according to the variables set at the top of the script file.

vitess/examples/local$ ./vttablet-up.sh

Output from vttablet-up.sh is below
Starting MySQL for tablet test-0000000100…
Starting vttablet for test-0000000100…
Access tablet test-0000000100 at http://localhost:15100/debug/status
Starting MySQL for tablet test-0000000101…
Starting vttablet for test-0000000101…
Access tablet test-0000000101 at http://localhost:15101/debug/status
Starting MySQL for tablet test-0000000102…
Starting vttablet for test-0000000102…
Access tablet test-0000000102 at http://localhost:15102/debug/status

After this command completes, refresh the vtctld web UI, and you should see a keyspace named test_keyspace with a single shard named 0. This is what an unsharded keyspace looks like.

If you click on the shard box, you’ll see a list of tablets in that shard. Note that it’s normal for the tablets to be unhealthy at this point, since you haven’t initialized them yet.

1. By launching tablets assigned to a nonexistent keyspace, we’ve essentially created a new keyspace. To complete the initialization of the local topology data, perform a keyspace rebuild

$VTROOT/bin/vtctlclient -server localhost:15999 RebuildKeyspaceGraph test_keyspace

1. Next, designate one of the tablets to be the initial master. Vitess will automatically connect the other slaves’ mysqld instances so that they start replicating from the master’s mysqld. This is also when the default database is created. Since our keyspace is named test_keyspace, the MySQL database will be named vt_test_keyspace.

$VTROOT/bin/vtctlclient -server localhost:15999 InitShardMaster -force test_keyspace/0 test-0000000100

example output:
master-elect tablet test-0000000100 is not the shard master, proceeding anyway as -force was used
master-elect tablet test-0000000100 is not a master in the shard, proceeding anyway as -force was used

Note: Since this is the first time the shard has been started, the tablets are not already doing any replication, and there is no existing master. The InitShardMaster command above uses the -force flag to bypass the usual sanity checks that would apply if this wasn’t a brand new shard.

After running this command, go back to the Shard Status page in the vtctld web interface. When you refresh the page, you should see that one vttablet is the master and the other two are replicas.

You can also see this on the command line:

$VTROOT/bin/vtctlclient -server localhost:15999 ListAllTablets test

example output:
test-0000000100 test_keyspace 0 master localhost:15100 localhost:33100 [] test-0000000101 test_keyspace 0 replica localhost:15101 localhost:33101 [] test-0000000102 test_keyspace 0 replica localhost:15102 localhost:33102 []

1. The vtctlclient tool can be used to apply the database schema across all tablets in a keyspace. The following command creates the table defined in the create_test_table.sql file:

Make sure to run this from the examples/local dir, so it finds the file.

vitess/examples/local $VTROOT/bin/vtctlclient -server localhost:15999 ApplySchema -sql "$(cat create_test_table.sql)" test_keyspace

1. Vitess uses vtgate to route each client query to the correct vttablet. This local example runs a single vtgate instance, though a real deployment would likely run multiple vtgate instances to share the load.

vitess/examples/local$ ./vtgate-up.sh

Stop Vitess cluster

1. The client.py file is a simple sample application that connects to vtgate and executes some queries. To run it, you need to either:

• Add the Vitess Python packages to your PYTHONPATH.
  or
• Use the client.sh wrapper script, which temporarily sets up the environment and then runs client.py.

vitess/examples/local$ ./client.sh

example output:
Inserting into master…
Reading from master…
(1L, ‘V is for speed’)
Reading from replica…
(1L, ‘V is for speed’)

1. Each -up.sh script has a corresponding -down.sh script to stop the servers.

vitess/examples/local$ ./vtgate-down.sh
vitess/examples/local$ ./vttablet-down.sh
vitess/examples/local$ ./vtctld-down.sh
vitess/examples/local$ ./zk-down.sh

1. Note that the -down.sh scripts will leave behind any data files created. If you’re done with this example data, you can clear out the contents of VTDATAROOT:

cd $VTDATAROOT
/path/to/vtdataroot$ rm -rf *

Refrence

Installation

Download Apache Solr from this link.
Unzip the Solr release and change your working directory to the subdirectory where Solr was installed

ls solr*
# solr-5.3.1.zip
unzip -q solr-5.3.1.zip
cd solr-5.3.1/

To launch Solr, run: bin/solr start -e cloud -noprompt

/solr-5.3.1:$ bin/solr start -e cloud -noprompt

Welcome to the SolrCloud example!

Starting up 2 Solr nodes for your example SolrCloud cluster.
...

Started Solr server on port 8983 (pid=8404). Happy searching!
...

Started Solr server on port 7574 (pid=8549). Happy searching!
...

SolrCloud example running, please visit http://localhost:8983/solr

/solr-5.3.1:$ _

You can load the Solr Admin UI in your web browser: http://localhost:8983/solr/

Indexing Data

Solr server is up and running, but it doesn’t contain any data. The Solr install includes the bin/post* tool in order to facilitate getting various types of documents easily into Solr from the start. We’ll be using this tool for the indexing examples below.

bin/post -c gettingstarted docs/

Here’s what it’ll look like:

/solr-5.3.1:$ bin/post -c gettingstarted docs/
java -classpath /solr-5.3.1/dist/solr-core-5.3.1.jar -Dauto=yes -Dc=gettingstarted -Ddata=files -Drecursive=yes org.apache.solr.util.SimplePostTool docs/
SimplePostTool version 5.3.1
Posting files to [base] url http://localhost:8983/solr/gettingstarted/update...
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
Entering recursive mode, max depth=999, delay=0s
Indexing directory docs (3 files, depth=0)
POSTing file index.html (text/html) to [base]/extract
POSTing file quickstart.html (text/html) to [base]/extract
POSTing file SYSTEM_REQUIREMENTS.html (text/html) to [base]/extract
Indexing directory docs/changes (1 files, depth=1)
POSTing file Changes.html (text/html) to [base]/extract
...
3248 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update...
Time spent: 0:00:41.660

The command-line breaks down as follows:

-c gettingstarted: name of the collection to index into
docs/: a relative path of the Solr install docs/ directory

Searching

Solr can be queried via REST clients, cURL, wget, Chrome POSTMAN, etc., as well as via the native clients available for many programming languages.

The Solr Admin UI includes a query builder interface – see the gettingstarted query tab at http://localhost:8983/solr/#/gettingstarted_shard1_replica1/query. If you click the Execute Query button without changing anything in the form, you’ll get 10 documents in JSON format (: in the q param matches all documents):

Search for a single term

To search for a term, give it as the q param value in the core-specific Solr Admin UI Query section, replace : with the term you want to find. To search for “foundation”:

curl "http://localhost:8983/solr/gettingstarted/select?wt=json&indent=true&q=foundation"

You’ll see:

/solr-5.3.1$ curl "http://localhost:8983/solr/gettingstarted/select?wt=json&indent=true&q=foundation"
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"indent":"true",
"q":"foundation",
"wt":"json"}},
"response":{"numFound":2812,"start":0,"docs":[
{
"id":"0553293354",
"cat":["book"],
"name":"Foundation",
...

The response indicates that there are 2,812 hits (“numFound”:2812), of which the first 10 were returned, since by default start=0 and rows=10. You can specify these params to page through results, where start is the (zero-based) position of the first result to return, and rows is the page size.

q=foundation matches nearly all of the docs we’ve indexed, since most of the files under docs/ contain “The Apache Software Foundation”. To restrict search to a particular field, use the syntax “q=field:value”, e.g. to search for foundation only in the name field:

curl "http://localhost:8983/solr/gettingstarted/select?wt=json&indent=true&q=name:foundation"

The above request returns only one document (“numFound”:1) – from the response:

...
"response":{"numFound":1,"start":0,"docs":[
{
"id":"0553293354",
"cat":["book"],
"name":"Foundation",
...

Phrase search

To search for a multi-term phrase, enclose it in double quotes: q=”multiple terms here”. E.g. to search for “CAS latency” – note that the space between terms must be converted to “+” in a URL (the Admin UI will handle URL encoding for you automatically):

curl "http://localhost:8983/solr/gettingstarted/select?wt=json&indent=true&q=\"CAS+latency\""

You’ll get back:

{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"indent":"true",
"q":"\"CAS latency\"",
"wt":"json"}},
"response":{"numFound":2,"start":0,"docs":[
{
"id":"VDBDB1A16",
"name":"A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM",
"manu":"A-DATA Technology Inc.",
"manu_id_s":"corsair",
"cat":["electronics", "memory"],
"features":["CAS latency 3,\t 2.7v"],
...

You can require that a term or phrase is present by prefixing it with a “+”; conversely, to disallow the presence of a term or phrase, prefix it with a “-“.

To find documents that contain both terms “one” and “three”, enter +one +three in the q param in the core-specific Admin UI Query tab. Because the “+” character has a reserved purpose in URLs (encoding the space character), you must URL encode it for curl as “%2B”:

curl http://localhost:8983/solr/gettingstarted/select?wt=json&indent=true&q=%2Bone+%2Bthree

Refrence

Solr Quick Start

Installation

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make
make install

You can edit redis configuration by edit etc/redis/redis.conf file.

Execute and test Redis

• Execute redis server
  $ redis-server

• Test redis server

$ redis-cli ping

PONG

Work with redis in CLI

1. Execute redis-cli

Add Key Value

SET key value

Get Value

GET key

Return : Value

Get list of all keys

key *

Get list of all values of All Keys

1. Get list with radis-cli

echo 'keys YOURKEY*' | redis-cli | sed 's/^/get /' | redis-cli

1. Get list with redis database dumper

rdd -v

Start, Stop and Restart Redis

/etc/init.d/redis-server restart
/etc/init.d/redis-server stop
/etc/init.d/redis-server start

Data Storage Type

With restart redis server, stored data will be accessibale

Reserved TCP Ports

Refrences

Redis Quick Start

Installation

1. Download From official website
2. Extract in /opt/ path
3. start server

./opt/neo4j/bin/neo4j

1. Execute CLI

./opt/neo4j/bin/neo4j-shell

The Property Graph Model

The property graph contains connected entities (the nodes) which can hold any number of attributes (key-value-pairs). Nodes can be tagged with labels representing their different roles in your domain. In addition to contextualizing node and relationship properties, labels may also serve to attach metadata—​index or constraint information—​to certain nodes.

Relationships provide directed, named semantically relevant connections between two node-entities. A relationship always has a direction, a type, a start node, and an end node. Like nodes, relationships can have any properties. In most cases, relationships have quantitative properties, such as weights, costs, distances, ratings, time intervals, or strengths. As relationships are stored efficiently, two nodes can share any number or type of relationships without sacrificing performance. Note that although they are directed, relationships can always be navigated regardless of direction.

Create Node

CREATE (john:Person:Author { name : 'John LE Cane' })
CREATE (lane:Person { name : 'Lane LE Cane' })
CREATE (tinker:Book { title : 'Tinker Tailor, Soldier, Spy' })

Create Relationship

CREATE (john)-[:WROTE]->(tinker)
CREATE (lane)-[:PURCHASED {date: “03-02-2011”}]->(tinker)

Query Data

MATCH (person:Person { name:'John LE Cane' })
RETURN person

Query All Data

MATCH (n)
RETURN n

Data Storage

After restart noe4j server, all saved data are still accessible

Reserved TCP Ports

References

Neo4j Manual
Neo4j Graph Database

Installation

1. Copy Download link in timetohide.me and download installation package.

2. Extract package in /opt/ path

3. Create /data/db/ directory
4. Start Server

sudo ./opt/mongo/bin/mongod

1. Execute Shell

sudo ./opt/mongo/bin/mongo

1. Show all Data Bases

show dbs

1. Use database

use db_name

1. Add document to collection

db.collection_name.insert({key: “value” , key2: “value2”})

1. Get document values

db.collection_name.find({key: “value”})

1. Show collections

show collections

1. Get all documents of collection

db.collection_name.find()

Data Storage

With restart server, data are still stored

Reserved TCP Ports

Refrences

Monogo DB web site