Aug 31, 2016

How to fix MySQL corrupted index

mysql> ALTER TABLE `hdgem` FORCE;

See also:

mysql optimize table vs alter table

[HDGEM] mysql optimize table vs alter table

To fix corrupted index problem etc in MySQL, it seems that 'alter table' is faster than 'optimize table'.

mysql> ALTER TABLE `hdgem` FORCE;
Query OK, 42097636 rows affected, 1 warning (27 min 20.81 sec)
Records: 42097636  Duplicates: 0  Warnings: 1

It is said that if you drop the index, alter the table, and then add index back, it will be even faster. But I have not tested it.

Posted By Blogger to HDGEM at 8/31/2016 11:17:00 AM

Aug 30, 2016

[HDGEM] Lightweight apps built on top of Java scale up and down easily.

Java is paying attention to the cloud, big data, and  microservices. 

Posted By Blogger to HDGEM at 8/30/2016 07:41:00 PM

[HDGEM] REST, which stands for REpresentational State Transfer, is the primary architectural style underlying the Web,

REST allows communication over the HTTP protocol with verbs such as GET, POST, PUT and DELETE.

Posted By Blogger to HDGEM at 8/30/2016 02:28:00 PM

[HDGEM] Apache Kafka is publish-subscribe based fault tolerant messaging system.

Kafka is fast, scalable and distributed by design.

Kafka is designed for distributed high throughput systems. Kafka tends to work very well as a replacement for a more traditional message broker. In comparison to other messaging systems, Kafka has better throughput, built-in partitioning, replication and inherent fault-tolerance, which makes it a good fit for large-scale message processing applications.

Posted By Blogger to HDGEM at 8/30/2016 02:25:00 PM

[HDGEM] What is a Messaging System?

A Messaging System is responsible for transferring data from one application to another, so the applications can focus on data, but not worry about how to share it. Distributed messaging is based on the concept of reliable message queuing. Messages are queued asynchronously between client applications and messaging system. Two types of messaging patterns are available − one is point to point and the other is publish-subscribe (pub-sub) messaging system. Most of the messaging patterns follow pub-sub.

Posted By Blogger to HDGEM at 8/30/2016 02:24:00 PM

[HDGEM] Twitter Streaming API

The "Twitter Streaming API" can be accessed in any programming language. The "twitter4j" is an open source, unofficial Java library, which provides a Java based module to easily access the "Twitter Streaming API". The "twitter4j" provides a listener based framework to access the tweets. To access the "Twitter Streaming API", we need to sign in for Twitter developer account and should get the following OAuth authentication details.
  • Customerkey
  • CustomerSecret
  • AccessToken
  • AccessTookenSecret

Posted By Blogger to HDGEM at 8/30/2016 02:22:00 PM

[HDGEM] Heron is a proven, production-ready, real-time stream processing engine from Twitter

Heron has been powering all of Twitter's real-time analytics for over two years. Prior to Heron, Twitter used Apache Storm, which they open sourced in 2011. Heron features a wide array of architectural improvements and is backward compatible with the Storm ecosystem for seamless adoption.

Posted By Blogger to HDGEM at 8/30/2016 02:17:00 PM

[HDGEM] How to check GTID MODE on MySQL

mysql> select @@GLOBAL.GTID_MODE;

Posted By Blogger to HDGEM at 8/30/2016 01:51:00 PM

[HDGEM] How to rename file in Git in case-insensitive file systems

git mv --force

In case-insensitive file systems changing the case of file names will be troublesome. This command makes git aware that the file has been renamed to

Posted By Blogger to HDGEM at 8/30/2016 10:49:00 AM

[HDGEM] Git tips

  • git diff-tree --no-commit-id --name-only -r <commit_ID>: see all the files that where changed in the commit <commit_ID>
  • git diff --name-only SHA1 SHA2: List all the files that have changed between commit SHA1 and SHA2

Posted By Blogger to HDGEM at 8/30/2016 10:48:00 AM

[HDGEM] How to Search for a String throughout all the history of a file in Git

Create a file called and paste this:

git rev-list --all $2 | (      while read revision; do          git grep -F $1 $revision $2      done  )
The command can then be executed like this: sh ./ string_to_search file_where_to_search

Posted By Blogger to HDGEM at 8/30/2016 10:47:00 AM

[HDGEM] Java mocking frameworks

Mocking: When using Java, dynamic mocking frameworks like EasyMockPowerMock and Mockito are very popular.

Posted By Blogger to HDGEM at 8/30/2016 10:39:00 AM

[HDGEM] Java Static Code Analysis Tools

FindBugs is a simpler static code analysis tool.
SonarQube is an entire platform for code quality analysis, syntax checking, and code metrics.

Posted By Blogger to HDGEM at 8/30/2016 10:35:00 AM

[HDGEM] Testing Frameworks for Java

Unit Testing

Top: JUnit
The de facto Java unit testing framework.


Top: Mockito
The clear favorite among Java mocking frameworks.

Automated Web Browser Testing

Top: Selenium
Selenium just automates the browsers. Developers usually use this tool along with testing frameworks to do large-scale web application testing.

Behavior-Driven Development (BDD) Testing

Top: Cucumber-JVM, Spock
Cucumber was originally a Ruby project but now has implementations for every major programming language. Spock supports the JVM language Groovy as well as Java. It could also handle much of what Mockito does.

Posted By Blogger to HDGEM at 8/30/2016 10:32:00 AM

[HDGEM] Java 8 has the fastest adoption rate of any new version to date.

This is likely due to long-awaited modern language features such as lambda expressions that cut down on some of the verbosity that Java is often criticized for.

Posted By Blogger to HDGEM at 8/30/2016 10:22:00 AM

[HDGEM] Avoid Constant Interface in Java

Java Constant interfaces have only static final data members declared in them without any methods.

It is suggested refactoring for this smell depends on kind of constants present in the constant interface:

  • the constants can get added as members in the class 
  • or can be rewritten as enums. 

Posted By Blogger to HDGEM at 8/30/2016 10:02:00 AM

[HDGEM] Smells are structures in code that violate design principles and negatively impact quality

"Refactoring for Software Design Smells: Managing Technical Debt", Girish Suryanarayana, Ganesh Samarthyam, Tushar Sharma, ISBN - 978-0128013977, Morgan Kaufmann/Elsevier, 2014. 

Posted By Blogger to HDGEM at 8/30/2016 10:00:00 AM

[HDGEM] With Java, anyone can easily create enterprise-level applications. It is object-oriented in nature and is suited for developing any modern web application. Not to mention it can also be used in creating web apps, games, desktop applications, mobile applications, etc.

Posted By Blogger to HDGEM at 8/30/2016 09:54:00 AM

[HDGEM] gRPC can help make connecting, operating and debugging distributed systems as easy as making local function calls

The framework handles all the complexities normally associated with enforcing strict service contracts, data serialization, efficient network communication, authentications and access control, distributed tracing and so on. gRPC along with protocol buffers enables loose coupling, engineering velocity, higher reliability and ease of operations. 

gRPC allows developers to write service definitions in a language-agnostic spec and generate clients and servers in multiple languages. Generated code is idiomatic to languages and hence feels native to the language you work on.

Posted By Blogger to HDGEM at 8/30/2016 03:04:00 AM

[HDGEM] Personalized Traffic Alerts From Google Maps

Google Maps pulls in data from several places, including crowd-sourced highway reports from its Waze service, to warn you what's up the road.

Posted By Blogger to HDGEM at 8/30/2016 02:53:00 AM

[HDGEM] It's critical to be able to develop and run systems efficiently and reliably at internet scale.

Building highly scalable, loosely coupled systems has always been tough. With the proliferation of mobile and IoT devices, burgeoning data volumes and increasing customer expectations, it's critical to be able to develop and run systems efficiently and reliably at internet scale. 

In these kinds of environments, developers often work with multiple languages, frameworks, technologies, as well as multiple first- and third-party services. This makes it hard to define and enforce service contracts and to have consistency across cross-cutting features such as authentication and authorization, health checking, load balancing, logging and monitoring and tracing, all the while maintaining efficiency of teams and underlying resources. 

It becomes especially challenging in today's cloud-native world, where new services need to be added very quickly and the expectation from each service is to be agile, elastic, resilient, highly available and composable.

Posted By Blogger to HDGEM at 8/30/2016 02:53:00 AM

[HDGEM] Facebook Blocks the Ad Blockers

The social media giant is fighting back against people who use ad blocking technology on their personal computers.

Posted By Blogger to HDGEM at 8/30/2016 02:53:00 AM

Aug 29, 2016

[HDGEM] A templating system for Java & JavaScript

Closure Templates simplify the task of dynamically generating HTML. They have a simple syntax that is natural for programmers. In contrast to traditional templating systems, in which you use one big template per page, you can think of Closure Templates as small components that you compose to form your user interface.

Closure Templates are implemented for both JavaScript and Java, so that you can use the same templates on both the server and client side. For the client side, Closure Templates are precompiled into efficient JavaScript.

Posted By Blogger to HDGEM at 8/29/2016 07:51:00 PM

[HDGEM] Jenkins Amazon ECS plugin

The Amazon EC2 Container Service Plugin can launch containers on your ECS cluster that automatically register themselves as Jenkins slaves, execute the appropriate Jenkins job on the container, and then automatically remove the container/build slave afterwards.

Posted By Blogger to HDGEM at 8/29/2016 06:24:00 PM

[HDGEM] CI tool with the largest market share in the space: Jenkins.

Posted By Blogger to HDGEM at 8/29/2016 06:09:00 PM

[HDGEM] New Surface Pro 3 firmware update to solve battery problems

Microsoft pushed out a firmware update to Surface Pro 3 devices on Monday to fix a problem that keeps the battery from charging all the way.

Posted By Blogger to HDGEM at 8/29/2016 06:07:00 PM

[HDGEM] Closure Tools from Google

The Closure tools help developers to build rich web applications with JavaScript that is both powerful and efficient. The Closure tools include: Closure CompilerClosure LibraryClosure Templates, and Closure Linter.

Posted By Blogger to HDGEM at 8/29/2016 05:15:00 PM

[HDGEM] Burnout is a situation where you mentally or physically collapse due to workload or related issues.

Burnout, defined in 1974 by Herbert Freudenberger, is mainly due to stress which can be caused by financial issues, heavy responsibilities and even failure to meet expectations. 

Around 62 percent of employees are said to be stressed and go through burnout at some stage. It is also one of the major reasons why employees quit.

Posted By Blogger to HDGEM at 8/29/2016 04:50:00 PM

[HDGEM] You need an Identity Store to which you can refer to verify your recognition of Apps and Users.

The Identity Store could be a database, but an LDAP server is the most popular solution. Active Directory is a popular LDAP implementation. 

In an LDAP server, you typically store usernames, passwords, digital certificates, some personal details and the organization groups to which Users belong. App IDs can also be stored here. 

An Identity Provider is software which is dedicated to managing the interaction with the Identity Store(s) for authentication and authorization purposes. 

Your API can function in this role though it is much more preferable to delegate this responsibility to the Identity Provider.

Posted By Blogger to HDGEM at 8/29/2016 03:43:00 PM

[HDGEM] Google treats social media posts as individual web pages when it comes to search.

If it's effectively a web page, the same rules of SEO should apply to them.

Posted By Blogger to HDGEM at 8/29/2016 02:31:00 PM

[HDGEM] Senior IT decision makers are right to be concerned about data security.

Data breaches now costing $400M or more.

Posted By Blogger to HDGEM at 8/29/2016 02:26:00 PM

[HDGEM] Wordpress security

WordPress is one of the most popular content management systems (CMS) in use today. According to Web Technology Surveys, WordPress, Joomla, and Drupal account for 70 percent of all CMS usage, with WordPress by far the most used of the three. But CMS platforms, by virtue of their open-source frameworks, are extremely vulnerable to cyber attacks, especially DDoS attacks.

Posted By Blogger to HDGEM at 8/29/2016 02:20:00 PM

[HDGEM] Token authentication is a more modern approach than server-side session IDs

Traditionally, applications have persisted identity through session cookies which rely on session IDs stored server-side. In this structure, developers are forced to create session storage that is either unique and server-specific, or implemented as a completely separate session storage layer.

Token authentication is a more modern approach to solve problems server-side session IDs can't. 

Using tokens in place of session IDs can lower your server load, streamline permission management, and provide better tools for supporting a distributed or cloud-based infrastructure.

In this method, tokens are generated for your users after they present verifiable credentials. The initial authentication could be by  username/password credentials, API keys or even tokens from another service.

Posted By Blogger to HDGEM at 8/29/2016 01:53:00 PM

[HDGEM] JWT is a secure and trustworthy standard for token authentication

JWTs allow you to digitally sign information with a signature and can be verified at a later time with a secret signing key.

Posted By Blogger to HDGEM at 8/29/2016 01:49:00 PM

[HDGEM] The microservices architecture is generally accepted as a better way to build apps these days.

Posted By Blogger to HDGEM at 8/29/2016 01:45:00 PM

[HDGEM] Microservices Architecture requirements

  • Services Contract
  • Exposing microservices from existing applications
  • Discovery of services
  • Coordination across services
  • Managing complex deployments and their scalability
  • Visibility across services

Posted By Blogger to HDGEM at 8/29/2016 01:41:00 PM

[HDGEM] Benefits of using memcached plugin on MySQL server

Advantages of using memcached include
  • Because all information is stored in RAM, the access speed is faster than loading the information each time from disk.
  • Because the "value" portion of the key-value pair does not have any data type restrictions, you can cache data such as complex structures, documents, images, or a mixture of such things.
  • If you use the in-memory cache to hold transient information, or as a read-only cache for information also stored in a database, the failure of any memcached server is not critical. For persistent data, you can fall back to an alternative lookup method using database queries, and reload the data into RAM on a different server.
  • The data held within a traditional memcached server is never stored on disk (only in RAM, which means there is no persistence of data), and the RAM cache is always populated from the backing store (a MySQL database). If a memcached server fails, the data can always be recovered from the MySQL database.

Posted By Blogger to HDGEM at 8/29/2016 01:19:00 PM

[HDGEM] How to enable Memcached plugin for MySQL server

To begin using the Memcached interface, you will need to take the following steps:
  1. Load the Memcached plugin at startup time by adding this option to your my.cnf file:

    plugin-load =

    You can also load it at runtime this way:

    mysql> install plugin daemon_memcached soname "";
  2. Load the Memcached-specific system schema, where the custom storage mappings between InnoDB and Memcached, along with some behavior configuration, are defined:

    mysql> source /usr/share/mysql/innodb_memcached_config.sql

Posted By Blogger to HDGEM at 8/29/2016 12:53:00 PM

[HDGEM] Oracle added NoSQL capabilities to the InnoDB engine in MySQL 5.6, providing a 9x improvement in transaction performance. Here's how to use the NoSQL features.

Oracle has implemented NoSQL interfaces to the MySQL database and MySQL Cluster. They bypass the SQL layer completely. Without SQL parsing and optimization, data can be written directly to MySQL tables up to nine-times faster, while maintaining ACID guarantees. Moreover, users can still run complex SQL queries across the same data set.

Posted By Blogger to HDGEM at 8/29/2016 12:42:00 PM

[HDGEM] Unlock the power of your emails

Email is an important part of how we get things done -- from planning an event with friends to organizing a trip to Paris. So much information is contained inside emails -- like the details of a dinner party or travel itinerary -- and so many emails require action -- like RSVP, or flight checkin.
By adding markup to the emails you send your users, you can make that information available across their Google experience, and make it easy for users to take quick action. Inbox, Gmail, Google Calendar, Google Search, and Google Now all already use this structured data. is a markup vocabulary that is standardized and managed as a collaboration of Google and other companies. By working with, we are creating an open standard, so that the markup you embed can be used by any email product that receives them.

Email Markup Tester from Google

Posted By Blogger to HDGEM at 8/29/2016 12:38:00 PM

[HDGEM] SEO tips: Submit all of your pages to the Google index - for free.

By using Google Search Console to submit your URLs, you help Google's web crawler do a more complete and efficient job of crawling your site. Search Console enables you to submit all of your pages to the Google index, and it's particularly useful for making sure that we know about all dynamically generated URLs or pages that are not adequately linked to on your site. But please note: submitting a page to the index will not guarantee inclusion or influence your PageRank, and isn't a replacement for creating compelling and useful content.

Posted By Blogger to HDGEM at 8/29/2016 12:30:00 PM

[HDGEM] Web testing

PhantomJS can be termed as yet another powerful tool for testing. It supports varied web standards and it is scriptable with JavaScript API which permits webpage capturing, pages manipulation, access to file system and page settings.

Selenium is an open source automation tool and definitely a great JavaScript automated testing framework which supports all operating systems and browsers such as Mozilla, Safari, Firefox, and IE. In addition, it runs multiple tests and supports programming languages such as Java, C#, Ruby, and Python.

Posted By Blogger to HDGEM at 8/29/2016 08:13:00 AM

[HDGEM] What challenge Mobile API usage?

Authentication, reliability, and performance constraints.

Posted By Blogger to HDGEM at 8/29/2016 04:05:00 AM

[HDGEM] Internet finance

The dramatic push into Internet finance opens up a revolutionary means of financing other than resorting to the capital market or conventional banks. It heralds an era in which all market participants can borrow and lend directly on the Internet with few information barriers. Booming Internet finance is closely associated with the development of so-called big data, which incorporates all sorts of information on a platform accessible to many people. The chance of a company defaulting on its loans can be more easily calculated by looking at the data of its past performance instead of just at its balance sheet.

Posted By Blogger to HDGEM at 8/29/2016 03:57:00 AM

Aug 27, 2016

[HDGEM] PhantomJS can be termed as a powerful tool for testing

 It supports varied web standards and it is scriptable with JavaScript API which permits webpage capturing, pages manipulation, access to file system and page settings.

Posted By Blogger to HDGEM at 8/27/2016 08:44:00 PM

[HDGEM] Unlike Apple’s FaceTime app, Google Duo can be used between the Android and iOS universes.

Posted By Blogger to HDGEM at 8/27/2016 05:30:00 AM

Aug 25, 2016

[HDGEM] A set of polyglot microservices

If we look at large scale systems from Google, Twitter, eBay, and Amazon, their architecture has evolved into something similar: a set of polyglot microservices.

Posted By Blogger to HDGEM at 8/25/2016 04:35:00 PM

[HDGEM] Kubernetes for Containers

Containers help abstract the machines from the code, so you can run any stack on any machine without having to explicitly configure that machine. Kubernetes automates the orchestration part of the story, so you can actually deploy and manage all these containers without having to SSH into machines.

Posted By Blogger to HDGEM at 8/25/2016 04:26:00 PM

[HDGEM] How to start a simple Python HTTP server from the Linux Shell command prompt

cd `mktemp -d` \      && echo '<html><body>Hello World</body></html>' >./index.html \      && python -m SimpleHTTPServer 8080
The following is just a test of the above command:
$ cd `mktemp -d` \
>     && echo '<html><body>Hello World</body></html>' >./index.html \
>     && python -m SimpleHTTPServer 8080 &
[1] 2470
~$ Serving HTTP on port 8080 ...
~$ curl http://localhost:8080 - - [25/Aug/2016 17:36:03] "GET / HTTP/1.1" 200 -
<html><body>Hello World</body></html>

Posted By Blogger to HDGEM at 8/25/2016 02:39:00 PM

[HDGEM] Google Cloud Shell - a tool for managing resources hosted on Google Cloud Platform

Google Cloud Shell is hosted on a virtual machine comes pre-installed with the Google Cloud SDK and other popular developer tools.

Your 5GB home directory will persist across sessions, but the VM is ephemeral and will be reset
approximately 20 minutes after your session ends. No system-wide change will persist beyond that.
Type "gcloud help" to get help on using Cloud SDK.

For more examples, visit and
Type "help" for direct help. Type "builtin help" to see Bash interpreter help.

Posted By Blogger to HDGEM at 8/25/2016 02:26:00 PM

[HDGEM] How to check microservices-based application is properly implemented

Microservices refers to an architectural style for developing applications. Microservices allow a large application to be decomposed into independent constituent parts, with each part having its own realm of responsibility. To serve a single user or API request, a microservices-based application can call many internal microservices to compose its response.
A properly implemented microservices-based application can achieve the following goals:
  • Define strong contracts between the various microservices.
  • Allow for independent deployment cycles, including rollback.
  • Facilitate concurrent, A/B release testing on subsystems.
  • Minimize test automation and quality-assurance overhead.
  • Improve clarity of logging and monitoring.
  • Provide fine-grained cost accounting.
  • Increase overall application scalability and reliability.

Posted By Blogger to HDGEM at 8/25/2016 02:19:00 PM

[HDGEM] ssl does not make connection via unix/Linux socket any more secure.

You are enforcing ssl conection via unix socket. Please consider switching ssl off as it does not make connection via unix socket any more secure.

Posted By Blogger to HDGEM at 8/25/2016 01:49:00 PM

[HDGEM] How to fix: find: Argument list too long

change the files into path.
Instead of:
$ find -name spam* path/*
$ find -name spam* path/

Posted By Blogger to HDGEM at 8/25/2016 01:29:00 PM

[HDGEM] How to fix MySQL Error dropping database (can't rmdir "test" errno: 39)

test folder is not empty. Remove test folder.

Posted By Blogger to HDGEM at 8/25/2016 12:09:00 PM

[HDGEM] How to Visualize Data from Many Sources

Kibana Explore & Visualize Your Data
See the Value in Your Data

  • Flexible analytics and visualization platform
  • Real-time summary and charting of streaming data
  • Intuitive interface for a variety of users
  • Instant sharing and embedding of dashboards

Easily visualize data pushed into Elasticsearch from Logstash, ES-Hadoop, Beats, or third-party technologies like Apache Flume, Fluentd, and many others.

Posted By Blogger to HDGEM at 8/25/2016 10:52:00 AM

[HDGEM] How to fix: cp: failed to preserve ownership

cp --no-preserve=mode

Posted By Blogger to HDGEM at 8/25/2016 10:41:00 AM

[HDGEM] How to fix: mv: failed to preserve ownership

mv is the wrong tool for this job; you want cp and then rm. Since you're moving the file to another filesystem this is exactly what mv is doing behind the scenes anyway, except that mv is also trying to preserve file permission bits and owner/group information. This is because mv would preserve that information if it were moving a file within the same filesystem and mv tries to behave the same way in both situations. Since you don't care about the preservation of file permission bits and owner/group information, don't use that tool. Use cp --no-preserve=mode and rm instead.

But if you don't care about the warning, mv actually does the move the files before complaining ownership problem.

Posted By Blogger to HDGEM at 8/25/2016 10:41:00 AM

[HDGEM] PageRank of individual page

All pages on your website have PageRank. This weight is accumulated from within your site, but its origin is other websites.

Sites linking to you pass PageRank, and the page they target on your site gets it. From there, the PageRank flows within your site based on your internal link structure.

Links from your most linked-to pages tend to carry the most weight.

Posted By Blogger to HDGEM at 8/25/2016 06:47:00 AM

[HDGEM] What Are Credit Card Skimmers?

Skimmers are are malicious card readers that grab the data off the credit card's magnetic stripe attached to the real payment terminals so that they can harvest data from every person that swipes their cards. The thief has to come back to the compromised machine to pick up the file containing all the stolen data, but with that information in hand he can clone cards or just break into bank accounts to steal money. Some skimmers don't prevent the ATM or credit card reader from functioning properly.

Posted By Blogger to HDGEM at 8/25/2016 05:15:00 AM

Aug 24, 2016

[HDGEM] Google Cloud SQL is a high-performance managed cloud MySQL database service, competitive with Amazon Aurora.

Cloud SQL Second Generation is an implementation of MySQL 5.7 on top of Google's Compute Engine and Persistent Disk. According to Google, Cloud SQL Second Generation runs seven times faster and has 20 times more storage capacity than its predecessor -- with lower costs, higher scalability, automated backups that can restore your database from any point in time, and 99.95 percent availability, anywhere in the world.

Posted By Blogger to HDGEM at 8/24/2016 04:27:00 PM

[HDGEM] Moving applications between cloud platform

Providing application portability is more than just making the app itself portable

Any variations in the software of cloud platform leads to performance issues. The source cloud and the target cloud are likely different in a number of ways: different hypervisors, different operating system support, different security, storage and network models and different database system support.

Management and monitoring tools that work in the source cloud may be unavailable in the target cloud too.

Posted By Blogger to HDGEM at 8/24/2016 04:14:00 PM

[HDGEM] Portable Lightweight Event-based Microservices on Google Cloud

Google Cloud Functions is a lightweight, event-based, asynchronous compute solution that allows you to create small, single-purpose functions that respond to cloud events without the need to manage a server or a runtime environment. Events from Google Cloud Storage and Google Cloud Pub/Sub can trigger Cloud Functions asynchronously, or you can use HTTP invocation for synchronous execution.

Cloud Functions are written in Javascript and execute in a standard Node.js runtime environment. You can take your Cloud Function and run it in any standard Node.js runtime which makes both portability and local testing a breeze. You can even deploy your Cloud Function directly from your GitHub or Bitbucket repository.

Posted By Blogger to HDGEM at 8/24/2016 01:52:00 PM

[HDGEM] How to start MySQL server

If for any other reason you cannot start MySQL server, the safe way to start it is:
sudo /bin/mysqld_safe --user=mysql &
sudo /bin/mysqld_safe --user=mysql --datadir=/path/to/existing-datadir

Posted By Blogger to HDGEM at 8/24/2016 12:51:00 PM

[HDGEM] mysql_upgrade — Check and Upgrade MySQL Tables

mysql_upgrade examines all tables in all databases for incompatibilities with the current version of MySQL Server. mysql_upgrade also upgrades the system tables so that you can take advantage of new privileges or capabilities that might have been added.

Posted By Blogger to HDGEM at 8/24/2016 12:32:00 PM

[HDGEM] MySQL slow shutdown

With a slow shutdown, InnoDB performs a full purge and change buffer merge before shutting down, which ensures that data files are fully prepared

To configure MySQL to perform a slow shutdown by setting innodb_fast_shutdown to 0. For example:
shell> bin/mysql -u root -p password --execute="set global innodb_fast_shutdown=0"
Shut down the MySQL server. For example:
shell> bin/mysqladmin -u root -p password shutdown

Posted By Blogger to HDGEM at 8/24/2016 12:04:00 PM

[HDGEM] Replication Compatibility Between MySQL Versions

MySQL supports replication from one release series to the next higher release series. For example, you can replicate from a master running MySQL 5.5 to a slave running MySQL 5.6, from a master running MySQL 5.6 to a slave running MySQL 5.7, and so on.

Posted By Blogger to HDGEM at 8/24/2016 11:47:00 AM

[HDGEM] performance penalty for enabling SSL in MySQL Connector/J

SSL in MySQL Connector/J encrypts all data (other than the initial handshake) between the JDBC driver and the server. There is a performance penalty for enabling SSL, the severity of which depends on multiple factors including (but not limited to) the size of the query, the amount of data returned, the server hardware, the SSL library used, the network bandwidth, and so on.

Posted By Blogger to HDGEM at 8/24/2016 11:23:00 AM

[HDGEM] How to Mirror Your Screen in Windows 10

Open the Start menu on your Windows 10 machine.

From here, you can just search for Connected Device Settings in the Start menu's search field to skip a few steps. If not, read on.

Click on Settings.

Select Devices.

Choose Connected Devices.

Click Add a Device.

Posted By Blogger to HDGEM at 8/24/2016 09:19:00 AM

[HDGEM] Percona Delivers Open Source In-Memory Storage Engine for Percona Server for MongoDB

Percona Memory Engine for MongoDB® is a 100 percent open source in-memory storage engine for Percona Server for MongoDB. Percona Memory Engine for MongoDB is based on WiredTiger, the in-memory storage engine used in MongoDB Enterprise Edition. It delivers extremely high performance and reduced costs for a variety of use cases, including application cache, sophisticated data manipulation, session management and more. Optimized for demanding workloads, Percona Memory Engine for MongoDB is the first open source in-memory storage engine to work with a MongoDB variant.

Posted By Blogger to HDGEM at 8/24/2016 06:47:00 AM

[HDGEM] Humans are the only creatures that make themselves suffer through negative emotions, particularly of self-judgment, regrets, and failed expectations

For the self-critical individual, chances are you are much harsher on yourself when you make mistakes than you would be to your colleagues, friends and family.

In today's connected world, we also constantly put ourselves up for comparison with others' constructed, perceived successes.

Posted By Blogger to HDGEM at 8/24/2016 06:38:00 AM

[HDGEM] Customized content delivered through bot messaging Berlin-based startup, Spectrum, enables publishers to share what is most relevant to each reader.

Customized content delivered through bot messaging
Berlin-based startup, Spectrum, enables publishers to share what is most relevant to each reader.

Posted By Blogger to HDGEM at 8/24/2016 06:32:00 AM

Aug 23, 2016

[HDGEM] 约克区日托中心和多间餐馆卫生不合格吃罚单

万锦市2860 Denison St 的Al-Habib Super Market
Aurora市24 Orchard Heights Blvd. 的Centra Food Market
万锦市9255 Woodbine Ave.的Ho Garden Chinese Restaurant(半岛餐厅)
Newmarket市17310 Yonge St.的Peekaboo Child Care Centre
Newmarket市 340 Eagle St. W.的Wild Wing

Posted By Blogger to HDGEM at 8/23/2016 06:50:00 PM

[HDGEM] Linux Filesystem

Everything in Linux is dumped together in a single filesystem hierarchy

Libraries in the Linux user space provide programs that interacted with the kernel (ifconfigsysctltuned-adm) and user-facing programs such as web servers or databases.

Users could inspect the /sbin or /lib directories and see all of the applications and libraries that support the operating system itself, or inspect the/usr/sbin or /usr/lib directory to see all of the user-facing programs and libraries

The problem with this model was that there was never complete isolation between operating system programs and business supporting applications. Programs in /usr/bin might rely on libraries which live in /lib. If an application owner needed to change something, it could break the operating system. Conversely, if the team in charge of doing security updates needed to change a library, it could break business facing applications.

That is why we need containers.

Posted By Blogger to HDGEM at 8/23/2016 02:43:00 PM

[HDGEM] How to check MySQL connection is using SSL or not ?

A client can determine whether the current connection with the server uses SSL by checking the value of the Ssl_cipher status variable. The value is nonempty if SSL is used, and empty otherwise. For example:
mysql> SHOW STATUS LIKE 'Ssl_cipher';
+---------------+--------------------+  | Variable_name | Value              |  +---------------+--------------------+  | Ssl_cipher    | DHE-RSA-AES256-SHA |  +---------------+--------------------+  1 row in set (0.00 sec)
For the mysql client, an alternative is to use the STATUS or \s command and check the SSL line:
mysql> \s  ...  SSL: Cipher in use is DHE-RSA-AES256-SHA  ...  Or:  mysql> \s  ...  SSL: Not in use  ...

Posted By Blogger to HDGEM at 8/23/2016 12:27:00 PM

[HDGEM] Google Cloud VPN securely connects your existing network to your Google Cloud Platform (GCP) network through an IPsec VPNconnection

Traffic traveling between the two networks is encrypted by one VPN gateway, then decrypted by the other VPN gateway. This protects your data as it travels over the Internet.
  • Cloud VPN provides an SLA of 99.9% service availability.
  • Cloud VPN supports site-to-site VPN. You can have multiple tunnels to a single VPN gateway.
  • Cloud VPN supports static routes for managing traffic between your instances and your existing infrastructure.
  • Cloud VPN supports both IKEv1 and IKEv2 using a shared secret (IKE pre-shared key).
  • Cloud VPN uses ESP in Tunnel mode with authentication. Cloud VPN does not support AH or ESP in Transport mode.

Posted By Blogger to HDGEM at 8/23/2016 10:32:00 AM

[HDGEM] IT security

• best practices, technologies, tools and trends relating to security, privacy, security architectures, security reference architectures, risk management (e.g. threat assessment, containment, remediation)
• security standards and best practices (e.g. ISO, SABSA, SANS)
• application security integration solutions using LDAP, PKI, IAA, IAM, DB authentication technologies

Posted By Blogger to HDGEM at 8/23/2016 09:37:00 AM

[HDGEM] Analytical skills

• research, analytical and problem-solving skills to review research and intelligence from multiple sources and develop options and responses to issues/incidents, concerns, and architectural design changes
• analytical and evaluative skills to assess client business requirements and develop/implement sound mechanisms

Posted By Blogger to HDGEM at 8/23/2016 09:35:00 AM

[HDGEM] Project management skills

• experience leading large-scale I&IT projects
• knowledge of project management processes and procedures, including budgeting, developing project plans, establishing priorities and critical paths, and monitoring project deliverables and milestones
• team leadership skills to provide direction, guidance and technical advice to project teams

Posted By Blogger to HDGEM at 8/23/2016 09:29:00 AM

Aug 22, 2016

[HDGEM] Users often choose OpenVPN because it’s open and secure

OpenVPN being de facto open source standard obviously has no known or even alleged vulnerabilities.

OpenVPN uses SSL/TLS encryption protocol, namely OpenSSL library.

OpenSSL library supports different encryption algorithms including Blowfish, AES, RC5 and 3DES.

Posted By Blogger to HDGEM at 8/22/2016 11:40:00 AM

[HDGEM] Open Source and Security

Surveillance is impossible in the open source world because any attempt to hide a tracking program inside a source code will be easily spotted. 

Things like Linux-based OS's and Chromium Web browser have become a lot more popular. Open Source has unintentionally become a must-have (or a least a must-try) for every security enthusiast.

Posted By Blogger to HDGEM at 8/22/2016 11:34:00 AM

[HDGEM] How much RAM to assign to MySQL

How you should tune your innodb_buffer_pool_size

70% to 80% is maybe a great start and rule of thumb.  You want to be sure the server has plenty of free RAM for the OS and other workload.  

Changing the InnoDB buffer pool requires a restart in 5.7- releases.

You tune the innodb_buffer_pool_size as large as possible without using swap when the system is running the production workload. 

MySQL 5.7 has online buffer pool resize feature which makes this an easier principle to follow. 

  • Seeing lots of free RAM (and/or filesystem cache usage)?  Turn the buffer pool up dynamically. 
  • Seeing some swap activity?  Just turn it down with no restart required.   

Posted By Blogger to HDGEM at 8/22/2016 10:56:00 AM

[HDGEM] OpenVPN is hugely popular and widely used

OpenVPN also allows following benefits:

  • Active Directory integration via LDAP
  • Multi factor authentication using Google's Authenticator app

Posted By Blogger to HDGEM at 8/22/2016 08:11:00 AM

Aug 21, 2016

[HDGEM] Laravel is a free, open-source PHP web framework

Laravel is created by Taylor Otwell and intended for the development of web applications following the model–view–controller (MVC) architectural pattern. Some of the features of Laravel are a modular packaging system with a dedicated dependency manager, different ways for accessing relational databases, utilities that aid in application deployment and maintenance, and its orientation toward syntactic sugar.

Laravel is regarded as one of the most popular PHP frameworks, together with Symfony2, Nette, CodeIgniter, Yii2 and other frameworks.

The source code of Laravel is hosted on GitHub and licensed under the terms of MIT License.

Posted By Blogger to HDGEM at 8/21/2016 03:34:00 PM

[HDGEM] Cisco would cut nearly 7 percent of its workforce, shifts focus from its legacy hardware towards higher-margin software.

Posted By Blogger to HDGEM at 8/21/2016 02:58:00 PM

[HDGEM] If you speak more than one language, your brain works differently to those who don't

Posted By Blogger to HDGEM at 8/21/2016 02:53:00 PM

Aug 20, 2016

[HDGEM] With as much as 90% of code in a typical project coming from third parties, it is evident that Security must have a voice in DevOps side to institute best practices that speed development while protecting against threats.

Posted By Blogger to HDGEM at 8/20/2016 04:48:00 PM

[HDGEM] containers promise to give Devs a way to Package Once, Deploy Anywhere, ensuring that Ops receives fully portable images that can run in virtually any environment.

Posted By Blogger to HDGEM at 8/20/2016 04:41:00 PM

[HDGEM] Choices other than JavaServer Faces (JSF) framework

Although JSF evolves as other web based software development, there are many other web frameworks available without its shortcomings, such as Node.js, ClojureScript and Angular.js applications that connect to RESTful backends.

Posted By Blogger to HDGEM at 8/20/2016 04:38:00 PM

[HDGEM] Facelets is a templating framework which is all static and provides no built in facilities for switching between faces asynchronously.

This is a massive drawback when attempting to develop sites that provide a single page interface, or a portlet based type of experience.

Posted By Blogger to HDGEM at 8/20/2016 04:33:00 PM

[HDGEM] More and more businesses are seeking to offer end users a feature-rich application with native-like functionality

JavaScript helps deliver the desired experience. Front-end and back-end as well as Dev and Ops are becoming ever more interdependent in the era of continuous integration and continuous delivery.

Posted By Blogger to HDGEM at 8/20/2016 04:29:00 PM

Aug 19, 2016

[HDGEM] How to set up SSH server to listen on multiple ports

To set up SSH server to listen on multiple ports, you can add multiple lines for ports, example:

# . . .

Port 22
Port 443

Posted By Blogger to HDGEM at 8/19/2016 01:49:00 PM

[HDGEM] Firefox allows you to set the proxy for just Firefox instead of setting a system-wide proxy.

Posted By Blogger to HDGEM at 8/19/2016 01:29:00 PM

[HDGEM] How to change the value of a variable while debugging Java code in NetBeans

To change the value of the primitive variables and Strings in-place in the Value column of Variables window. To change values of Object variables, you can use "Evaluate Expression..." under menu Debug option.

Posted By Blogger to HDGEM at 8/19/2016 08:30:00 AM

Aug 18, 2016

[HDGEM] Following TCP streams using Wireshark

If you are working with TCP based protocols it can be very helpful to see the data from a TCP stream in the way that the application layer sees it. Perhaps you are looking for passwords in a Telnet stream, or you are trying to make sense of a data stream. Maybe you just need a display filter to show only the packets of that TCP stream. If so, Wireshark's ability to follow a TCP stream will be useful to you.

Simply select a TCP packet in the packet list of the stream/connection you are interested in and then select the Follow TCP Stream menu item from the Wireshark Tools menu (or use the context menu in the packet list). Wireshark will set an appropriate display filter and pop up a dialog box with all the data from the TCP stream laid out in order.

Posted By Blogger to HDGEM at 8/18/2016 04:47:00 PM

[HDGEM] How to capture emails with Wireshark

  1. Get wireshark 
  2. Into filter enter smtp click Apply 
  3. When you get filtered lines click right mouse button on one of them and select 'Follow TCP stream'.
  4. For base64 encoded contents, you can convert them to text by tools or online service. 

Posted By Blogger to HDGEM at 8/18/2016 04:44:00 PM

[HDGEM] "Nothing helps calibrate reality [more] than the honest perceptions of those who work closest to you." Jay Samit

"Nothing helps calibrate reality [more] than the honest perceptions of those who work closest to you."
Jay Samit

Posted By Blogger to HDGEM at 8/18/2016 07:09:00 AM

[HDGEM] Moz are killing off Moz Content and Followerwonk

Moz trims product line to focus on search, lays off 28% of staff

Posted By Blogger to HDGEM at 8/18/2016 07:07:00 AM

Aug 17, 2016

[HDGEM] Hadoop-related projects at Apache

  • Ambari™: A web-based tool for provisioning, managing, and monitoring Apache Hadoop clusters which includes support for Hadoop HDFS, Hadoop MapReduce, Hive, HCatalog, HBase, ZooKeeper, Oozie, Pig and Sqoop. Ambari also provides a dashboard for viewing cluster health such as heatmaps and ability to view MapReduce, Pig and Hive applications visually alongwith features to diagnose their performance characteristics in a user-friendly manner.
  • Avro™: A data serialization system.
  • Cassandra™: A scalable multi-master database with no single points of failure.
  • Chukwa™: A data collection system for managing large distributed systems. 
  • HBase™: A scalable, distributed database that supports structured data storage for large tables. 
  • Hive™: A data warehouse infrastructure that provides data summarization and ad hoc querying. 
  • Mahout™: A Scalable machine learning and data mining library. 
  • Pig™: A high-level data-flow language and execution framework for parallel computation.
  • Spark™: A fast and general compute engine for Hadoop data. Spark provides a simple and expressive programming model that supports a wide range of applications, including ETL, machine learning, stream processing, and graph computation.
  • Tez™: A generalized data-flow programming framework, built on Hadoop YARN, which provides a powerful and flexible engine to execute an arbitrary DAG of tasks to process data for both batch and interactive use-cases. Tez is being adopted by Hive™, Pig™ and other frameworks in the Hadoop ecosystem, and also by other commercial software (e.g. ETL tools), to replace Hadoop™ MapReduce as the underlying execution engine.
  • ZooKeeper™: A high-performance coordination service for distributed applications.

Posted By Blogger to HDGEM at 8/17/2016 08:19:00 PM