Enterprise Systems

Enterprise System 2jpg

 

Enterprise Systems are application packages which allow an organisation integrate their entire IT system together.  In essence, company-wide access to required business knowledge becomes available, rather than independent systems which don’t talk to each other and duplicate data, or manual input of data to reports & spreadsheets.  They support the business processes and flow of information in an organisation, while also enabling easier reporting and analytics within the company.  The final goals being efficient running of the company, increasing work quality, saving employee time, and possible reducing costs.

Real World.

At this point many would think ‘Sounds good, but why would my business need it?’

Let me give you a real-life example.  My Dad was looking for to purchase some Potash for his fruit trees recently.  He rang around the local garden shops & agricultural stores to find the best price-by-weight.  One particular store was about half the price of the others, and even though it was 10-miles further out of town, he considered it worth the drive.  (Considering we were  only talking €8 for 2.5kg versus €8 for 1.2kg it might not have been a decision I would have made!)

Decision made, he drove the extra ten miles to the agricultural store.  He ordered his Potash, paid for it, was issued with an Invoice & receipt, went down to the store to hand in his invoice, and was told they had none in stock!  He then had to return to the Reception area, get a refund for the purchase he wasn’t able to make, then leave for another store.  Not a happy customer.

If we look at the interactions that took place between customer and retailer, how many places could we have made the interaction better?

Now let’s look at the scenario where the company had an Enterprise system in place.  In this case all the companies information would be stored on one database, this would include stock levels & pricing.  The person who took the original phone call could have seen the stock levels at the time they were checking the price.  As a result they could have informed my Dad they didn’t have the stock in place, and he wouldn’t have had an wasted drive & the bad customer experience.

An even better result would be that the company had agreed minimum stock levels saved in the system, say 10 bags of Potash.  When the 10th last bag in stock is sold, the system automatically sends through an order to their suppliers, and the cash accounts would reflect the order.  No interaction required by the store staff to log a low stock level.  And more importantly, no customers turned away unhappy.  In addition, the managers of the store would always have available analytics on the use of the product, and be able to perfect the minimum stock levels for each product to ensure they don’t have too much stock to hand at all times.

Sounds like a better system all round ?!

Details

Enterprise System

An Enterprise System is essentially a centralised database, and a number of software modules which integrate together.  All the different divisions of the company write to the database.  This data is then available for the other software modules to use within the organisation.

In our picture above the Sales Forecast from the Sales & Marketing department can inform the Production Schedules for Manufacturing & Production department.  This in turn will inform the Material requirements, which require financing from the Finance & Accounting department.  All of these conversations require no passing of paper reports between department.

Company Management can obtain point-in-time information on company operations as required.

Enterprise Systems for large vendors include SAP NetWeaver & Oracle Fusion.  With options available on the market for Small & Medium companies, or those who want to use cloud services.

All systems include customisable tables, so you can customise the system to the way your compnay works, e.g. you can organise by location, product range, etc.  If the customisable tables are not enough to get the system to work the way you want it to, you could code changes to the system.  However, this is inadvisable.  Firstly because there are tricky systems to change – you could affect other areas without knowing.  But Secondly these applications are built around best practice principles, and you should look at your business processes first for change.

Opinion

I think enterprise systems are brilliant.  Having one reliable and accurate source of company data is the Holy Grail for somebody who has worked with multiple legacy systems.  However, there in lies the major problem I see with these – legacy systems & the willingness to do away with old systems.  It’s fine for a small company who want to grow into the future – not too much data or too many business processes to bring across from old systems.  Or even for major multi-national companies with money behind them, e.g. Coca-Cola switching to a SAP enterprise system to leverage their buying power for raw materials (Laudon & Laudon, 2014, pg 372).  However, if your company has been around for a while, you probably have a number of systems, and business processes in place.  Are you willing to spend the time & money doing a full analysis of their systems?  Identifying what can go, what needs to change, and what is absolutely required for your company going forward?  Only then can you even think about switching to an enterprise system.  If you’re not going to take this on board fully, I see it as wasted money, as inevitably, any system will be customised to the point where it no longer functions as well as it can do.

Second major issue to be looked at is security & data access.  This is obviously a general concern for all IT systems.  However, it’s worth spending extra time considering the Security Policy of Enterprise Systems.  In a case where HR had their own Personnel databases, they could control who had access to employee records in-house.  In an Enterprise system, the Personnel data is held on the centralised database.  This means Identity Management needs to be put in place to identify who can have access to which data, and what level of data access they can have, e.g. Read, write, or update access.  The concern I would have is that you’ll need somebody who knows how to customise this access on the staff.  Meaning the company needs IT resources or else pay for consultants to do the customisation.  This is an extra cost that needs to be taken into account going forward with Enterprise Systems.

Overall, Enterprise systems seem a logical way to look after a companies data & provide great advantages to an organisation.

_______________________

https://en.wikipedia.org/wiki/Enterprise_system

Laudon K & Laudon J. (2014) ‘Management Information Systems: Managing the Digital Firm’ Pearson, Edinburgh.

Hadoop

hadoopHadoop is an open source suite of programs, procedures & tools created by Apache Software Foundation.  They are designed to facilitate the analysis of very large datasets of Structured & non-Structured data.

Hadoop makes it possible to work on thousand of nodes involving many terabytes, or even petabytes, of data.  This makes analysis of big data substantially easier.  It also has rapid data transfer rates among nodes, meaning if one goes down, you can transfer it’s work to another.  This gives it a high degree of fault tolerance.  Reducing the risk of failures slowing processing.

So, how does it work?  At its simplest, Hadoop takes a large big data analysis problem, and breaks it down into smaller problems.  It then distributes the smaller problems to inexpensive distributed computers or servers for parallel processing.  It then combines the results for easy analysis, or further processing.

History:

Hadoop was inspired by Google’s MapReduce white paper.  This paper described why they created MapReduce – to be able to index the huge increase in data on the internet.  Hadoop was released in 2005 by Apache Software Foundation, as an open source product.  Although one of the more interesting facts about its history is the name.  Doug Cutting was one of the creators of Hadoop, and it was named after his child’s stuffed toy elephant.  Digital Marketers all over the world are despairing as a result…

hadoop bed time

Hadoop Ecosystem:

Hadoop was originally composed of the core components of MapReduce, and HDFS (Hadoop Distributed File System).  However, as further components were added for specific needs, the number of components increased.  These are now generally referred to as the Hadoop Ecosystem.

Let’s look at the base modules:

Hadoop distributed file system (HDFS)

File System that keeps track of data across large number of linked storage.  It can be accessed by any computer using a supported operating system.  It will accept any type of data, you just put it in the cluster, and leave it there until you decided how you want to process it.

Hadoop actually supports many different file systems, for example the Amazon Web Service integrates Hadoop with it’s own S3 file system.  But HDFS is the Hadoop version.

MapReduce

This is the default data processing system.  It’s Java-Based.  As Hadoop is not a relational database, or indeed a database at all, you cannot use SQL to get answers from the data.  As a result, you use NoSQL.

Hadoop Common

Tools and libraries needed for other Hadoop modules.

YARN

Manages resources of systems which store the data & run the analysis.

 

Hadoop ecosystem

There are now also a number of additional modules you can use for different processing on top of these modules.  For example,

 Advantages:

– Very Flexible. Easy to scale up or down on inexpensive computers or servers.

– Popular.  It’s widely used in Big Data industry, and as a mature product offering support is available.

– Free as it’s open source.  Also means, if software experts make enhancements, it’s fed back into the development community for general use.

– It will process any amount of data – Petabytes and above.  The data can be in any form – Structures, non-structures, emails, patents, voicemails, etc.

Disadvantages:

– In it’s basic state, Hadoop can be complex to use.  As a result, commercial versions are being created with simplified use.  For example, Cloudera, Hortonworks, MapR, etc.  You’ll pay for support & consultation with many of these.

– MapReduce is not a good match for all analysis.  It’s quite a complicated piece to work with, and can often only handle one problem at a time.  This means you either need experts to run it, or look into alternative options to run it, for example HIVE (which has a broader range of skilled practitioners available).

– Hadoop is a more mature option, when it comes to open source Big Data processing.  But it’s not necessarily the best.  A lot is being said about Spark these days, which is quickly finding it’s place in industry.  Just because Hadoop was out first, does not make it the best for your requirements.  Maybe you need both?  But it’s a decision that needs thought put into it.  What do you actually want to process and how?

Opinion

First off, a disclaimer: I’ve never used Hadoop.  My opinion is formed from articles on the internet, and the content of text books.  So, feel free to disagree.

It seems to me, Hadoop was the best option at the time, and it still does some things well.  However, it’s difficult to use, and as a result, people are adding bits on top which go against the original premise.  For example, Putting relational technology (SQL) on top of Hadoop, which is neither a database nor relational, seems daft.  The whole concept of big data concerns, is that existing models of processing can’t cope.  Why then try to restrict it backwards with processing we’re up to speed with?  If training, and a lack of experts is the problem, well then train them up.  IT professionals of any reasonable skill level are adept at learning new computer languages, and platforms.  They are inherently interested in learning new ways to do cool things.  (And yes that includes myself.)  If Big data processing is the means to the future of data analytics, then be open to the alternative ways of working that come with it.

Secondly, it’s all very well loading data into your Hadoop cluster until you’re ready to work with it.  But that brings security risks & expense if you’re on cloud computing services.  It’s hard to see how data governance can be enforced if you’re not even sure where your data is, or what is in it.  We’re told again, and again, that computer piracy is on the up, and to do as much as you can to protect your data, especially customer & core organisational data.  However, the risk with big data is we get overwhelmed with it, and as a result don’t go to the same rounds of protecting it.  This is more a big data issue, than a Hadoop issue.  But reading these articles, it’s seen as a major advantage of Hadoop to be able to throw anything into the Hadoop cluster until you’re ready to use it.  One even mentioned thinking of Hadoop as a big bucket!  As long as you remember you need to lock that big bucket in a safe, inside a safe, inside a castle with a big moat around it, you may be OK.

I’m a big fan of Open Source programs.  Apart from the fact they’re free, there is generally a whole community of developers willing to help you use them to the fullest.  The downside is they’re often not as user friendly as those created to make profit.  I believe this is part of the issue with Hadoop.  The original developers came up with a great solution.  All the add-ons since then have either been a solution to another problem, using the core functionality of Hadoop as its foundations.  Or have made more user-friendly options available.  At it’s core it’s a great solution to the processing of very large data.  We shouldn’t forget the core value of that solution when overtaken by all the add-ons.

hadoop

Data

Data Star Trek

What is Data?

According to Wikipedia, Data is “uninterpreted information”.  It then goes on to ask what type of Data are you interested in?  Options include ‘a fictional android from Star Trek’, a book by Euclid, a moth, a British Drum & Bass musician, and a ‘non-governmental organisation founded by Bono’! 1   For this blog I’m interested in Computing Data, although the android lived an interesting life too …

Data is raw facts- numbers, text, images, symbols, etc.  On it’s own data means nothing.  Data needs to be interpreted, grouped & processed, in order to become usable information.  After all Colonel Mustard, in the library, with the Candlestick, doesn’t mean anything.  However, if you know you’re playing Cluedo, it makes all the difference!

Data_and_information

What is information?

This leaves us with the understanding that information is interpreted data.  Information is what we use to make decisions & decide future actions.

If you’ve ever applied for a mortgage, you know the amount of financial information you’re required to hand into the bank – Salary information; Bank Accounts, balances & statements for 6 months; Any loans; Details of current rent, etc.  At it’s basic level it’s just figures on a page.  The bank then groups this data together into Earnings &  Outgoings.  This information is then used to decide if a Mortgage approval will be given, and for home much.

In order for the decision to be useful, the processed data must be:

If we go back to our mortgage example, if the information is 2 years old, it’s not useful to enable us make an informed decision on a customers current financial situation.  We may not know about a new gambling addiction that would rule out approving a mortgage.  In the same way, if we believe a customer has an outstanding loan for €1,000, but it’s actually €10,000, that makes a big difference to a decision.  Lastly, if the bank only records one salary, rather than two for a couple, the financial situation will not be complete and the decision cannot be correct.

In days gone by, all of the above mortgage decision was done on paper in your bank, where they knew your name.  If they were still unsure, you’d get an important member of the community to give you a reference, i.e. priest, doctor, etc.!  These days, it’s all on computers, for good or ill!  As an IT professional, I’d say it’s progress, and point out the amazing things we can do now with IT.  But then you couldn’t have put a country in debt to billions the old way …

What is Knowledge?

Knowledge it the understanding of the context of information.  How to interpret the information.

 

Data_Info_Knowledge

Reverting back to our mortgage example for the last time, looking at one persons Income & Expenditure only gives the scoring team so much information to make a decision.  However, if they have the same information for 1,000 previous scoring decisions, and the outcomes of the mortgages granted, then they will be able to look at the information with experienced eyes.  So, looking at your expenditure and seeing a BetFair or PaddyPower online account regularly used, that is just information.  However, if the last 20 people they gave a mortgage to with the same outgoings history got into financial difficulties, then that’s a concern.  Only by having that context on the information, can the bank acquire the knowledge to look for the red flags.

 

We now know that data is the building block for knowledge.  So, how does a computer store data?

That is to be continued in later blog…

DATA

 

 

——————————-

1. https://en.wikipedia.org/wiki/Data_%28disambiguation%29