Data Diary #2

Data Visuals : Dreams unlimited!!

Visuals are there everywhere, from primitive Ajanata, Ellora caves to the huge Bulletin board of the metropolis. As some cultural theorists say, we live in the world of images. And it is un-disputable that images speak fancier than words, if not louder. And as days progress, data is becoming more and more ubiquitous and visible in all the quadrants of life. It is not that all these data has been born into our networks all of a sudden. Instead, we are becoming more and more conscious about the possibility of sharing and transforming the data. 

Hence these two universes are colliding and co-creating a new paradigm called data visualization. By the time you analyse the thoughts, you can visit the below page to know the mushrooming tools in the data visualization space. This shows the demand in this space. Periodic Table of Data Visualization Tools

Data visualization is a field with exciting features and powers in Web Intelligence. Web Intelligence is slowly transforming the Business Intelligence space all together. Data Visualization as a strategy stands very much coherent to IBM's thought of simplifying life. Gone are the days when we need the expertise of complex queries to mine the data from huge data stores. Now the imperative is to leave the complexities to a selected few data scientists and to empower the rest of the world to focus on the beauty of the data. By the time I finish this introduction on data visualization, at least some of us may be started exploring or experiencing in our daily work or social media life. 

In Public internet we come to see its capabilities through tools such as Klout, Cloze, Vizify, Orange etc.Thus Social Business is no longer confined to the interfaces of text analytics and streaming hash tags. It has become more intuitive and info-graphic in nature. Image analytics may be a tough ask for the present computing circuits. But any day, we can see it happen. Visualizaton need not be confined to data alone as meta data can also be visualized. At then end of the day, only your imagination is your boundary!

Twitter: @gokulgaze

Note: All the views are purely personal ...

Data Diary #1

Monsters in Memory !

Can you imagine idle java processes eating away 70% of your Operating system RAM share. In our Content management system built on a propriety product suite, we were taken by surprise to realize that it is a baggage from the product architecture. However, I must not point fingers at the vendor as they have mentioned in their product manuals to disable the non-essential services to be disabled in the early phase it self. But we have left with no such choice now. May be this kind of elasticity & flexibility is desirable any product in the days forward.

Our content management system follows an event based consumer - subscriber model for sharing the events across different content related services such as Workflow, We found that our event subsystem and the associated small relational database related Java Processes are alive all the time. That must be the rationale behind keeping these processes alive.

Along with it, we could find a Servlet and Search agent built on Java API's alive all the time. All the villains are identified. But not able to keep them behind the bars as they are essential for the system's standard behavior and as-is status. This makes me think and highlight that Content management systems are still memory and event intensive model. I must say that this +ECM has derived lot of base model from the standards of Version Control systems and relational data models. What made it unique was the OS level coupling it has achieved by adding a lot of modifications on the OS registries and device drivers at the firmware level .. Anyway I am just curious what will these bunch of monsters will do tomorrow as the story continues. Data Diary and Data Dilemma continues ...

+Gokul Alex

PS: All views are my own

Social Business Diary #1

Business & IT Convergence ...

Fake Clouds: A late realization

As we know, with more and more of Business and +Information Technology  convergence, cloud computing need to meet the rationale of Software Economics. We can see that the dynamics between 'Money' and 'Data' is still a perplexing question. There are some who says 'Data' is 'Money'. But people are realizing that only with a clear cut model for software economics they can evaluate the hypes and marketing frameworks that mystify each software invention.

Deadly Sins of Business Agility

What should be understood by the term 'Business Agility' ? How +Information Technology  can transform your Business to an agile environment? It all revolves around your understanding of key drivers of growth and dampening factors that divert your focus. This link will tell us that, without a clear cut and forward looking +business architecture , Business Agility will remain a jargon for ever.

Product Development in India

As the product development spirits are gearing up in India, and as we conceive more and more innovation within the design space, it is time for us to think of a broader strategy for Product Life-cycle Management( +PLM ). And it is where +IBM can help you with a vast arrays of methods and tools ...

Social Business -3 Mistakes to avoid

Social Business is a buzz word with lot of meaning and nothingness within. It will end up as a hollow pot if we don't use our intelligence and understand our business. So this post pronounces that the basics of Business remains the same even we embrace social space!

+Gokul Alex

PS: All views are my own

Social Media Analysis and the Method of Difference

The next generation business strategies will be heavily dependent on the information generated by the social media demography of the netizens. The doubt is only about the capability of present business models to leverage the potentials of this data. Best of our efforts are in the direction of analyzing the 'Big Data' generated by the multitude of social media networks. We constantly try to devise an algorithm that can approximate the behavioral pattern of the data generation in these networks. But it is high time to realize that the approximations and content matching algorithms involves lot of wastage of computational logic and infrastructure. 

Instead of depending on the predictive computational models, it is time to devise difference engines. Our prediction filters should be replaced by difference engines. The more difference we have in the model and the actual data, the more information we are receiving from the social media networks. For example, if we have a template for a user profile in Facebook based on his activities for the last one month, if we get totally surprising data on the next day, we should welcome it with all happiness and curiosity. This difference should be analyzed and integrated back to the template. This feedback mechanism will generate more and more non-linear data about this user. 

A further interesting point for the difference engines would be to locate the reasons of the increasing complexity of social networks. The first question is, if at all the complexity is increasing. If it is the case, what is the reason, is it caused by the accumulation of data, or is is due to the increase of connections between the nodes in the social network graphs.

Wikipedia presents one defintion of social media as this:

"In general, social networks are self-organizing, emergent, and complex, such that a globally coherent pattern appears from the local interaction of the elements that make up the system."

But are social networks self-organizing. Is data produced and consumed in social networks, self organized?

Note:: This post is a draft in progress ...

Some Notes on SQL Query Optimization

This post is just inspired by some reflections and doubts triggered by a session on SQL (Structured Query Language) Query Optimization I attended today. 

The speaker had mentioned that earlier versions of Oracle used Rule based Optimization whereas the present versions of Oracle RDBMS uses Cost Based Optimization (CBO). 

From my limited knowledge of SQL I had doubts about the way SQL will display results when we use GROUP BY clause along with aggregate functions.

Indeed, I am intrigued when the DBA said that JOINs are less costly than SUB QUERY. 

Indexes in RDBMS always amused me. In my earlier project, where we had plenty of data integrity issues caused by PRIMARY KEY CONSTRAINT related exceptions. Somehow or other I had managed to eliminate them by clearing the PRIMARY KEY related indexes.

In today's session, a DBA with SQL Server expertise mentioned about Clustered Indexes. It was a wild character to my limited knowledge of database systems!

The presenter mentioned that indexes are mainly used when we are selecting rows from columns. This statement was really impressive to me. 

The DBA highlighted that each developer can play his / her part in SQL optimization and query fine tuning by analysing the EXPLAIN PLAN table. They should ask for access to EXPLAIN PLAN table. Then they should run a kind of util command that will add EXPLAIN PLAN table on their own schema. This table will help us to optimize the queries before the real execution. 

SQL profiler is a tool from Oracle that can help in analyzing the Queries. I need to know more about it. 

Finally, DBA mentioned that sometimes the third party tools and applications that we install on the RDBMS can spoil the entire performance of the system. The Data backup tools, Search and Indexing engines etc are some examples for this. The main reason for the impact on the performance is the share of CPU resources demanded by these tools. These tools end up eating up the OS resources making RDBMS engine weak and unstable.

Let me end this brief note with the expectations that I can carry forward this exploration further and further ...