Sharding in Database

Video reference for database sharding:
http://www.youtube.com/watch?v=FwJ7bJaan7Y

1>Vertical sharding ; eg. user can have many contacts and user data can grow enormously while contacts data can grow more enormously so Use table's data is stored in one machine(called shard say shard1) and Contact is stored in another machine(shard2) .
2>Horizontal sharding : eg. User's table can have millions and billions of data so now sharding is seprated depending uopn the user alphabet lik users with starting name from A-E store in shard1 and user's from F-G store in shard2 etc and a mappping table is created to say A-E users are stored in shard1 and F-G users are stored in shard2.So query should first lookup to this mapping table and then to actual shards.

>>Types of 

Relational vs Non relational databases in short

>>Relational Databases

• Scale Vertically : Because of the way they are structured, relational databases usually scale vertically – a single server has to host the entire database to ensure reliability and continuous availability of data.
• 
  

>>NON relational Databases:

• Scale vertically :meaning that to add capacity, a database administrator can simply add more commodity servers or cloud instances. The database automatically spreads data across servers as necessary.
• Replication(high availability) :Most NoSQL databases also support automatic replication, meaning that you get high availability and disaster recovery without involving separate applications to manage these tasks. The storage environment is essentially virtualized from the developer's perspective.
• Integrated Caching:increased performance, faster accessibility 

Mongodb info , NoSQL vs SQL

>>Mongodb CMS mongopress
http://www.mongopress.org/key-features/
>>http://www.zdnet.com/rdbms-vs-nosql-how-do-you-pick-7000020803/
>>http://www.mongodb.com/learn/nosql
>>

NoSQL vs. SQL Summary

	SQL Databases	NoSQL Databases
Types	One type (SQL database) with minor variations	Many different types including key-value stores, document databases, wide-column stores, and graph databases
Development History	Developed in 1970s to deal with first wave of data storage applications	Developed in 2000s to deal with limitations of SQL databases, particularly concerning scale, replication and unstructured data storage
Examples	MySQL, Postgres, Oracle Database	MongoDB, Cassandra, HBase, Neo4j
Data Storage Model	Individual records (e.g., "employees") are stored as rows in tables, with each column storing a specific piece of data about that record (e.g., "manager," "date hired," etc.), much like a spreadsheet. Separate data types are stored in separate tables, and then joined together when more complex queries are executed. For example, "offices" might be stored in one table, and "employees" in another. When a user wants to find the work address of an employee, the database engine joins the "employee" and "office" tables together to get all the information necessary.	Varies based on database type. For example, key-value stores function similarly to SQL databases, but have only two columns ("key" and "value"), with more complex information sometimes stored within the "value" columns. Document databases do away with the table-and-row model altogether, storing all relevant data together in single "document" in JSON, XML, or another format, which can nest values hierarchically.
Schemas	Structure and data types are fixed in advance. To store information about a new data item, the entire database must be altered, during which time the database must be taken offline.	Typically dynamic. Records can add new information on the fly, and unlike SQL table rows, dissimilar data can be stored together as necessary. For some databases (e.g., wide-column stores), it is somewhat more challenging to add new fields dynamically.
Scaling	Vertically, meaning a single server must be made increasingly powerful in order to deal with increased demand. It is possible to spread SQL databases over many servers, but significant additional engineering is generally required.	Horizontally, meaning that to add capacity, a database administrator can simply add more commodity servers or cloud instances. The database automatically spreads data across servers as necessary
Development Model	Mix of open-source (e.g., Postgres, MySQL) and closed source (e.g., Oracle Database)	Open-source
Supports Transactions	Yes, updates can be configured to complete entirely or not at all	In certain circumstances and at certain levels (e.g., document level vs. database level)
Data Manipulation	Specific language using Select, Insert, and Update statements, e.g. SELECT fields FROM table WHERE…	Through object-oriented APIs
Consistency	Can be configured for strong consistency	Depends on product. Some provide strong consistency (e.g., MongoDB) whereas others offer eventual consistency (e.g., Cassandra)

Collection of free books , resources etc

http://blog.fogus.me/2013/12/27/the-best-things-and-stuff-of-2013/

Software Engineering

>>Software Engineering Body Of Knowledge(SWEBOK)
http://www.computer.org/portal/web/swebok

xampp apache mysql installation problems

• Change xampp apache port no.

         go to C:\xampp\apache\conf\httpd.conf find Listen and change it to port other than 80...
         http://complete-concrete-concise.com/web-tools/how-to-change-the-apache-port-in-xampp

• root@localhost with password 'No' Access denied problem.

         change password 

         C:\xampp\phpMyAdmin\Config.inc.php 

         find $cfg['Servers'][$i]['password']      = '12345';       and restart apache server  

• delete service apache2.2 when apache doesnot restart and try starting apachen again.

          sc delete "apache2.2"
          http://www.howtogeek.com/howto/windows-vista/how-to-delete-a-windows-service-in-vista-or-xp/

>>upgrade xampp php version
http://stackoverflow.com/questions/2154762/upgrading-php-in-xampp-for-windows

car2go info

github link on how to use car2go api
https://github.com/hanseartic/FreeCars/blob/master/FreeCars/Providers/Car2Go.cs

JAVA real all files in a folder

import java.io.File;

        String directory = System.getProperty("user.dir");
System.out.println(directory);
File file = new File(directory+"\\Munich\\");
for(File item : file.listFiles()){
System.out.println(item.getAbsolutePath());
}

Useful links for rails

• setting up database connection in rails application

http://www.packtpub.com/article/working-with-rails-setting-up-and-connecting-to-a-database

• Diaspora link for setup :  https://wiki.diasporafoundation.org/Installation/Ubuntu/Precise#Database_setup

Network commands to see ssl certificates

• openssl s_client -connect twitter.com:443
• openssl s_client -connect twitter.com:443 -CAfile certdata.crt

 Copy the part beginning with ––-BEGIN CERTIFICATE––- up to ––-END CERTIFICATE––- into a file twitter.com.cert. Then issue the following command: openssl x509 -in twitter.com.cert

-text. The result is a text representation of the certificate content. 

sample ssl certificate

twitter.com

Identity: twitter.com

Verified by: VeriSign Class 3 Extended Validation SSL CA

Expires: 10/05/14

Subject Name

1.3.6.1.4.1.311.60.2.1.3: #13025553

1.3.6.1.4.1.311.60.2.1.2: #130844656C6177617265

2.5.4.15: #131450726976617465204F7267616E697A6174696F6E

serialNumber (Serial Number): 4337446

C (Country): US

2.5.4.17: #14053934313037

ST (State): California

L (Locality): San Francisco

STREET (Street): 795 Folsom St, Suite 600

O (Organization): Twitter, Inc.

OU (Organizational Unit): Twitter Security

CN (Common Name): twitter.com

Issuer Name

C (Country): US

O (Organization): VeriSign, Inc.

OU (Organizational Unit): VeriSign Trust Network

OU (Organizational Unit): Terms of use at https://www.verisign.com/rpa (c)06

CN (Common Name): VeriSign Class 3 Extended Validation SSL CA

Issued Certificate

Version: 3

Serial Number: 1E 22 C7 37 A3 91 5E 3F AB 65 C4 B5 A4 1C AE 46

Not Valid Before: 2012-04-10

Not Valid After: 2014-05-10

Certificate Fingerprints

SHA1: C3 1F 6D 53 92 F2 CB 48 0A 42 79 8C 1F BE 70 82 1D D8 82 51

MD5: 27 74 1B 9F 5E D4 67 0B FD FD ED 49 5A AE AF 8F

Public Key Info

Key Algorithm: RSA

Key Parameters: 05 00

Key Size: 2048

Key SHA1 Fingerprint: DA 05 9C 75 A6 AE 93 54 48 CB F9 FA 24 15 83 30 63 74 F8 B0

Public Key: 30 82 01 0A 02 82 01 01 00 BE E9 77 B4 C2 85 91 74 53 6D 5A 09 FC 54 23 37 54 1E 6E 2C 89 6C 5C 43 AE FC C5 17 AB 0C 8D A1 5E 64 E1 89 4C 95 2A 71 D2 09 E4 81 FA 00 C1 5C 6E 2A A1 11 D2 11 CC 2A EC 9D 58 84 5B 1C B0 AC FA A3 89 2D B1 62 EA 56 8C 06 49 63 5B 5B DD 99 6E D8 CE 1A 44 C3 DF 17 69 3B C1 6A 7E 07 2C E4 BA B5 C8 AE 75 EE 7A DB B4 4A 7A 39 71 16 72 85 AF D7 9B E8 3F 8C 15 ED 9F 47 8C 23 FA BF 93 75 32 6A CF 37 5F 3B C0 33 07 5B 59 9E 26 AC 50 FF B2 E2 50 B8 15 40 C5 55 FD 4E AA 2C E8 54 78 DA 65 FF 4E 82 AB 14 9C 53 56 DF 9D B7 A9 E5 2A 21 50 6F B3 E9 DF 02 51 A3 D7 E7 91 21 1A D0 0F CD 73 49 00 70 D2 92 23 60 48 A9 0F AF 55 12 27 D6 22 5E EB 2A 9A 37 0F 0A 14 5F 91 3E FB 91 13 A3 5A 7C 60 57 EA 2E 70 5F 93 8D AF 88 B2 DF 1A 70 AC B6 B2 00 A2 76 15 32 B9 70 96 1B 8E 1B 21 FF 02 03 01 00 01

Subject Alternative Names

DNS: www.twitter.com

DNS: twitter.com

Critical: No

Basic Constraints

Certificate Authority: No

Max Path Length: Unlimited

Critical: No

Subject Key Identifier

Key Identifier: B5 78 90 46 79 AF 6E E7 5D 40 48 C4 47 C6 F0 E0 28 C1 31 84

Critical: No

Key Usage

Usages: Digital signature
Key encipherment

Critical: No

Extension

Identifier: 2.5.29.31

Value: 30 39 30 37 A0 35 A0 33 86 31 68 74 74 70 3A 2F 2F 45 56 53 65 63 75 72 65 2D 63 72 6C 2E 76 65 72 69 73 69 67 6E 2E 63 6F 6D 2F 45 56 53 65 63 75 72 65 32 30 30 36 2E 63 72 6C

Critical: No

Extension

Identifier: 2.5.29.32

Value: 30 3B 30 39 06 0B 60 86 48 01 86 F8 45 01 07 17 06 30 2A 30 28 06 08 2B 06 01 05 05 07 02 01 16 1C 68 74 74 70 73 3A 2F 2F 77 77 77 2E 76 65 72 69 73 69 67 6E 2E 63 6F 6D 2F 72 70 61

Critical: No

Extended Key Usage

Allowed Purposes: Server Authentication
Client Authentication

Critical: No

Extension

Identifier: 2.5.29.35

Value: 30 16 80 14 FC 8A 50 BA 9E B9 25 5A 7B 55 85 4F 95 00 63 8F E9 58 6B 43

Critical: No

Extension

Identifier: 1.3.6.1.5.5.7.1.1

Value: 30 6E 30 2D 06 08 2B 06 01 05 05 07 30 01 86 21 68 74 74 70 3A 2F 2F 45 56 53 65 63 75 72 65 2D 6F 63 73 70 2E 76 65 72 69 73 69 67 6E 2E 63 6F 6D 30 3D 06 08 2B 06 01 05 05 07 30 02 86 31 68 74 74 70 3A 2F 2F 45 56 53 65 63 75 72 65 2D 61 69 61 2E 76 65 72 69 73 69 67 6E 2E 63 6F 6D 2F 45 56 53 65 63 75 72 65 32 30 30 36 2E 63 65 72

Critical: No

Extension

Identifier: 1.3.6.1.5.5.7.1.12

Value: 30 60 A1 5E A0 5C 30 5A 30 58 30 56 16 09 69 6D 61 67 65 2F 67 69 66 30 21 30 1F 30 07 06 05 2B 0E 03 02 1A 04 14 4B 6B B9 28 96 06 0C BB D0 52 38 9B 29 AC 4B 07 8B 21 05 18 30 26 16 24 68 74 74 70 3A 2F 2F 6C 6F 67 6F 2E 76 65 72 69 73 69 67 6E 2E 63 6F 6D 2F 76 73 6C 6F 67 6F 31 2E 67 69 66

Critical: No

Signature

Signature Algorithm: SHA1 with RSA

Signature Parameters: 05 00

Signature: 0A A0 F3 5A 00 0C 47 44 FB F0 A5 9B 84 38 DD AB C5 4E 4A 6E 1E 4A 58 6E ED 1B 0E F4 95 86 83 12 9F 5D 9B D0 20 2D 22 06 3B 4A 0F A4 23 4F D3 D0 F6 9A 0C 3C 72 EA 51 26 F1 80 2E FC D2 CD BE 05 E1 20 46 0A C5 94 3B B1 4A CA AD F6 3E 6A F7 4C 3E E9 98 1A 45 9E 43 4C 44 7F 46 4B 6C 60 9F A7 D0 C0 F9 CC 31 1A 06 5A C5 3D 07 76 12 E1 49 93 AF 97 0D C5 DE F5 9E CD 4B AB 89 46 1A 71 12 FA 01 6D 44 CC 2D EE 53 6C E3 E9 3D 33 DA FE F9 5D AF 2E E8 0C 18 E9 8A 80 A0 C2 42 C9 1D 76 6D CE EC 1A 44 68 0F 46 A4 DB B0 DD BC FE 7A AA 1E 4D D2 D8 C7 1C C2 35 77 A2 09 B3 44 C0 3B 6E BF 49 52 3B 62 54 17 12 CB 4A 1E 4F 62 27 36 A9 A7 06 7F D0 76 49 B1 EA 57 EE 32 31 BE 5C 72 28 A1 9B F0 C4 06 8D 20 B0 C7 C6 D8 D8 F6 D8 68 B7 B4 AE 49 36 22 31 F4 68 C5 25 1D 6E FB 0E 93 40 2B 8D

Document oriented database ( mongodb )

Useful links about mongodb talk on youtube.
http://www.youtube.com/watch?v=w5qr4sx5Vt0

http://www.youtube.com/watch?v=liQzIsFnCr0

Learn typo3 CMS

http://typo3.org/documentation/videos/

join IRC cloud and get connected to developers anytime.

https://www.irccloud.com/

( http://typo3.org/support/irc-chat/ )

The following channels exist on Freenode and are available for open discussion on their respective topics:

• #typo3 - General chat and code issues surrounding implementation of TYPO3 websites
• #typo3-flow - Discussion and developer assistance concerning the FLOW3 framework
• #typo3-neos - For chat and guidance relating to the upcoming TYPO3 Neos
• #typo3-cms - A channel for developing TYPO3 CMS

• steps to transfer old typo3 cms to new version 6.1

http://typo3.3.n7.nabble.com/moved-a-10-years-old-TYPO3-3-5-to-6-1-td243121.html

I just moved contents from my first TYPO3 site, a 10 years old TYPO3 
3.5, to the new 6.1 
It was 2003, 10 years ago! 
Everything works like a charm! 

If someone is interested here the "how to": 

Copy pages and tt_content tables from old site to new TYPO3 db (dump and 
restore with drop tables) 

Copy uploads/pics uploads/media from old sites to fileadmin/pics 
fileadmin/media (or make a symbolic link from uploads/pics to 
fileadmin/pics) 

Install tool -> compare tables to adjust all fields 

Install tool -> migrate wizard -> Migrate all file relations from 
tt_content.image and pages.media 

Install tool -> migrate wizard -> Migrate file relations of tt_content 
"uploads" 

>>http://typo3.toaster-schwerin.de/typo3_english/2013_05/msg00167.html

Enable ping command in windows 7

http://www.howtogeek.com/howto/windows-vista/allow-pings-icmp-echo-request-through-your-windows-vista-firewall/

or run cmd with Run as administrator and try ping google.com , it worked for me when typing ping in command with user prevelidge prompted with error that 'ping' command is unrecognized command.

hosting and configuring zencart e-commerce in cpanel.

1>login to www.buddhistthings.com/cpanel
2>Navigate to File manager select webroot buddhistthings.com
3>use file zilla to upload in directory public_html
4>Navigate to mysql option and create database zencart and also username and password for database.  DON'T forget to 

•    add username to database with security privileged (option is available below create database and user  option)because if you don't do so zencart will propmt with database conncetion error.eg.Sorry!

        There seems to be a problem connecting to our database. Please give us a few minutes to           remedy the problem. Thank you.
4>configure includes>>configure.php and admin>>includes>>configure.php with database settings and other firle paths
Don't forget to 

•  generally change database server name  to localhost rather than buddhistthings.com,DB username to username created above similarly password and and database name .
• change define('DIR_FS_CATALOG', '/home/cpanelusername/public_html/zencart')

proxy pattern

http://ramj2ee.blogspot.in/2013/11/proxy-design-pattern.html

A remote proxy provides a local representative for an object in a different address space.

A virtual proxy creates expensive objects on demand.

A protection proxy controls access to the original object. Protection proxies are useful when objects should have different access rights.

A smart reference is a replacement for a bare pointer that performs additional actions when an object is accessed.

Provide a surrogate or placeholder for another object to control access to it. Proxy means ‘in place of’ or  ‘Representing’ or ‘in place of’ or ‘on behalf of’ There are four common situations in which the Proxy pattern is applicable.

1. A virtual proxy is a placeholder for "expensive to create" objects. The real object is only created when a client first requests/accesses the object.eg google map
2. A remote proxy provides a local representative for an object that resides in a different address space. This is what the "stub" code in RPC and CORBA provides.eg webservice call
3. A protective proxy controls access to a sensitive master object. The "surrogate" object checks that the caller has the access permissions required prior to forwarding the request.
4. A smart proxy interposes additional actions when an object is accessed. Typical uses include:  
   • Counting the number of references to the real object so that it can be freed automatically when there are no more references (aka smart pointer)
   • Loading a persistent object into memory when it's first referenced
   • Checking that the real object is locked before it is accessed to ensure that no other object can change it

Agent Based Modeling Software

http://en.wikipedia.org/wiki/Comparison_of_agent-based_modeling_software


Cougaar (acronym of Cognitive Agent Architecture) is a Java-based architecture for building large-scale distributed applications based on agents.
Altreva Adaptive Modeler is a software application for creating agent-based financial market simulation models for the purpose of forecasting prices of real world market traded stocks or other securities.

Framsticks is a 3D freeware Artificial Life simulator.  Framsticks allows users to design organisms or manually edit the living genetic code of an organism

Try mongodb

http://try.mongodb.org/
http://docs.mongodb.org/manual/tutorial/install-mongodb-on-windows/
>

DBCollection help

db.foo.count()

db.foo.dataSize()

db.foo.distinct( key ) - eg. db.foo.distinct( 'x' )

db.foo.drop() drop the collection

db.foo.dropIndex(name)

db.foo.dropIndexes()

db.foo.ensureIndex(keypattern,options) - options should be an object with these possible fields: name, unique, dropDups

db.foo.find( [query] , [fields]) - first parameter is an optional query filter. second parameter is optional set of fields to return.

e.g. db.foo.find( { x : 77 } , { name : 1 , x : 1 } )

db.foo.find(...).count()

db.foo.find(...).limit(n)

db.foo.find(...).skip(n)

db.foo.find(...).sort(...)

db.foo.findOne([query])

db.foo.getDB() get DB object associated with collection

db.foo.getIndexes()

db.foo.group( { key : ..., initial: ..., reduce : ...[, cond: ...] } )

db.foo.mapReduce( mapFunction , reduceFunction , )

db.foo.remove(query)

db.foo.renameCollection( newName ) renames the collection

db.foo.save(obj)

db.foo.stats()

db.foo.storageSize() - includes free space allocated to this collection

db.foo.totalIndexSize() - size in bytes of all the indexes

db.foo.totalSize() - storage allocated for all data and indexes

db.foo.update(query, object[, upsert_bool])

"

DBCollection help

show collections              show collections in current database

db.help()                     help on DB methods

db.foo.help()                 help on collection methods

db.foo.find()                 list objects in collection foo

db.foo.save({a: 1})           save a document to collection foo

db.foo.update({a: 1}, {a: 2}) update document where a is 1

db.foo.find({a: 1})           list objects in foo where a == 1

it                            result of the last line evaluated; use to further iterate

About Web RTC

https://developer.mozilla.org/en-US/docs/Web/API/Navigator.getUserMedia

Canvas in html5
https://developer.mozilla.org/en/docs/Web/API/HTMLCanvasElement

AudioContext in mozilla
https://hacks.mozilla.org/2013/07/web-audio-api-comes-to-firefox/

http://www.webrtc.org/reference/getting-started

chat
http://stackoverflow.com/questions/14134090/how-is-a-webrtc-peer-connection-established

http://www.codeproject.com/Articles/515192/Video-Conferencing-using-WebRTC

web RTC video conferencing slides
http://html5videoguide.net/presentations/WebDirCode2012/#page5

http://dev.w3.org/2011/webrtc/editor/webrtc.html

http://www.w3.org/TR/webrtc/#simple-peer-to-peer-example

https://www.webrtc-experiment.com/docs/webrtc-for-newbies.html

Publish papers in European journal/Read awarded research papers

http://www.journals.elsevier.com/european-journal-of-operational-research

Vehicle Routing Problem(VRP)

Vehicle routing problem

From Wikipedia, the free encyclopedia

A figure illustrating the vehicle routing problem

The vehicle routing problem (VRP) is a combinatorial optimization and integer programming problem seeking to service a number of customers with a fleet of vehicles. Proposed by Dantzig and Ramser in 1959,^[1] VRP is an important problem in the fields of transportation, distribution and logistics. Often the context is that of delivering goods located at a central depot to customers who have placed orders for such goods. Implicit is the goal of minimizing the cost of distributing the goods. Many methods have been developed for searching for good solutions to the problem, but for all but the smallest problems, finding global minimum for the cost function is computationally complex.

Contents

  [hide] 

• 1 Overview
• 2 See also
• 3 References
• 4 Further reading
• 5 External links

Overview[edit]

Several variations and specializations of the vehicle routing problem exist:

• Vehicle Routing Problem with Pickup and Delivery (VRPPD): A number of goods need to be moved from certain pickup locations to other delivery locations. The goal is to find optimal routes for a fleet of vehicles to visit the pickup and drop-off locations.
• Vehicle Routing Problem with LIFO: Similar to the VRPPD, except an additional restriction is placed on the loading of the vehicles: at any delivery location, the item being delivered must be the item most recently picked up. This scheme reduces the loading and unloading times at delivery locations because there is no need to temporarily unload items other than the ones that should be dropped off.
• Vehicle Routing Problem with Time Windows (VRPTW): The delivery locations have time windows within which the deliveries (or visits) must be made. In computational complexity theory, this problem is known to be NP-hard.^[2]
• Capacitated Vehicle Routing Problem (with or without Time Windows): CVRP or CVRPTW. The vehicles have limited carrying capacity of the goods that must be delivered.
• Vehicle Routing Problem with Multiple Trips (VRPMT): The vehicles can do more than one route.

Several software vendors have built software products to solve the various VRP problems. Numerous articles are available for more detail on their research and results.

Although VRP is related to the Job Shop Scheduling Problem, the two problems are typically solved using different techniques.^[3]

See also[edit]

• Travelling salesman problem

References[edit]

1. Jump up^ Dantzig, George Bernard; Ramser, John Hubert (October 1959). "The Truck Dispatching Problem". Management Science 6 (1): 80–91.
2. Jump up^ Beck, Prosser, Selensky, 2003; p.2 left
3. Jump up^ Beck, J.C.; Prosser, P.; Selensky, E. (2003). "Vehicle routing and job shop scheduling: What’s the difference". Proceedings of the 13th International Conference on Artificial Intelligence Planning and Scheduling.

Some vendors implementing these concept

https://www.routist.com/index?secure=true