Barriers or excuses for not moving to cloud

Since the time I have been going hard on learning and experimenting the cloud computing with the leading IaaS provider - AWS - I have always been thinking what stops CIOs from moving to the cloud. In the due course of reading papers, talking to people, I got some points what they call barriers:

• Performance and scalability - If a CIO thinks that one cannot move to cloud and be as scalable and high performing, I have to say that he/she needs to be educated on some of the fundamentals on cloud computing. Auto-scaling (Up and Down) automatically - THE biggest asset! How can someone prefer leaving their infrastructure idle or shutting down services to provide more server resources in case the demand peaks to take the Autoscaling opportunity provided by cloud providers such as AWS?? Scientific researchers, NASA live telecast from MARS Rover, banking industries, blue chips companies - please - they all are in cloud albeit hybrid model but they are and dont say that their cloud infrastructure is not performing as good. 
• Infrastructure "Type" - Really?? At the end of the day do you want "T5", IBM XIV, Solaris BLADE, etc etc or you are looking for the same PERFORMANCE?? Are you really looking for managing those infrastructure no matter how many times you have to have the outages or do you want highly scalable, available and durable infrastructure with bare minimum utility based costing model - NO MATTER WHETHER THEY RUN YOUR SERVICES ON CHEAP INFRASTRUCTURE! Do you really care??
• Security - Funny! Its one of the MAIN reasons moving to the cloud. The level of security AWS can provide, pretty much impossible to achieve unless you move away from your CORE business which may be selling a product and just manage secure infrastructure :) Please read 100s of pages from AWS Security White Paper - Each and Every service is SECURE - aside from app level and firewall rules level security that the coimpanies will provide on top! 

From what I can understand, companies will gradually move away from on-premise infrastrucure ONLY when they will see their competitors gaining extra-ordinary advantage all of a sudden, but Alas! it might already have been late move from them. The whole idea is NOT TO WORRY ABOUT your infrastrcture - that takes MOST of your IT time, resources and money and FOCUS on your CORE business, develop new services, products, deploy them within seconds for your customers, add more agility in your IT environment. Make your infrastrucure more secure, scalable, available with multi zone DR planning already in place within minutes if not seconds. Compare this with on-premise infrastructure maintainance - Dealing with vendors, upgrading Oracle, paying millions (Even for idle infrastructure), waiting for approval - rings a bell??

Setting up Amazon EC2 with MySQL, glassfish and access from within Proxy

Check out the Android OpMedia Application intro
http://www.youtube.com/watch?v=Ta2FIHiTIn0&feature=youtu.be

Download link from Google Play Store
https://play.google.com/store/apps/details?id=com.kiikka.opmedia.android.media.activities&hl=en#!

Find out whether IBM AIX file systems are LUNs

Scenario: A developer needs to know whether the file storage system is local or pointing to a LUN. For instance, the file systems he/she works on is /u02. Does it map to a local storage or SAN?

Steps:
1. Issue a mount command to first list out the mounted LOGICAL and/or PHYSICAL file systems and their mount points.

Example:
$ mount
node    mounted  mounted over    vfs       date             options    


-------- ---------------  ---------------  ------ ------------ ---------------
         /dev/hd4         /                jfs2   28 Oct 03:11 rw,log=/dev/hd8
         /dev/hd2         /usr             jfs2   28 Oct 03:11 rw,log=/dev/hd8
         /dev/hd9var      /var             jfs2   28 Oct 03:11 rw,log=/dev/hd8
         /dev/hd3         /tmp             jfs2   28 Oct 03:11 rw,log=/dev/hd8
         /dev/hd1         /home            jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /dev/hd11admin   /admin           jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /proc            /proc            procfs 28 Oct 03:16 rw            
         /dev/hd10opt     /opt             jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /dev/livedump    /var/adm/ras/livedump jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /dev/fslv00      /audit           jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /dev/u01lv       /u01             jfs2   28 Oct 03:16 rw,log=/dev/hd8
         /dev/u03lv       /u03             jfs2   28 Oct 03:16 rw,cio,log=INLINE
         /dev/u06lv       /u06             jfs2   28 Oct 03:16 rw,cio,log=INLINE
         /dev/u02lv       /u02             jfs2   28 Oct 03:16 rw,cio,log=INLINE
         /dev/u07lv       /u07             jfs2   28 Oct 03:16 rw,cio,log=INLINE
tggfilesvr1 /D/TempNFS       /mnt/tggfilesvr1 nfs3   28 Oct 03:16 rw,bg,hard,intr,sec=sys

Clearly, /dev/hd* are the physical drives. However, u**lv seem to be logical volumes created by grouping one or more physical drives together.

2. Issue a command lslv to list the details of a logical volume.
Example:
$ lslv -l u01lv
u01lv:/u01
PV                COPIES        IN BAND       DISTRIBUTION
hdisk7            184:000:000   34%           000:064:000:093:027
hdisk0            184:000:000   34%           000:064:000:101:019


$ lslv -l u02lv
u02lv:/u02
PV                COPIES        IN BAND       DISTRIBUTION
hdisk2            1450:000:000  25%           150:375:200:375:350


Clearly, LV u01lv is made of hdisk7 and hdisk0 whereas u02lv (mounted on u02) is mapped to hdisk2.

3. Now we know that u02 file in question is pointing to physical drive hdisk2. In order to make sure whether it is in fact pointing to SAN, issue following command:

$ lsattr -El hdisk2 |grep lun_id
lun_id          0x2000000000000                        Logical Unit Number ID           False

If there is no SAN involved there will not be any output. Else, there will be one as above.


Thanks for reading.

Integration with File System

Scenario: An ESB hooks up to an application via a file system - mounted or local. It polls the file generated by the system. Once available, it reads it and processes it in order to send it to final destination. Challenge: 1. A partially generated file may get picked up by the ESB. 2. Non-transactional behavior. Options:

Time based polling: This approach is for ESB to wait for a few minutes before polling the file system. Assumption: We know the maximum size of the file that the application will generate and we are 100% sure that the file is completely generated after x number of minutes.

Risk: Suppose we polled for a file at a time t. So, the next polling starts at a time t + x assuming that the next file will be completely written to the disk by the time. However, there was a failure in the application generating the file and by the time it started generating the file again, it is already too late for it to finish writing the file completely by t+x. Thus, ESB gets a half-cooked file.

Conclusion: I think this approach is very risky and likely to generate data inconsistency very often in productive environment.

Size based polling: This approach is for ESB to poll several times before it "intelligently" concludes that the file has been written completely by looking at the file size. Assumption: After n number of polling for the same file, if the file size does not grow, we are 100% sure that the file is completely written by the application. 

Risk: There are chances when while writing the file, the application may fail. If the Exception/Compensation is not properly handled by the application, the partially generated file will not be deleted by the application and will stay there. The ESB after polling n number of times, will assume that the file is completely written because its size remains the same. However, actually it is not complete and might be missing its trailer or header.

Conclusion: This is a much better option that the first. However, even in this case there are some problems as mentioned above. So, unless there is a mechanism where the application gets notified by ESB that it has processed x number of records or there was a problem in this file (Which is difficult in file based Fire-And-Forget kind of asynchronous approach), or a proper exception handling in place within the application, there is always a great risk of losing the data - esp if the file structure and parsing is important from ESB point of view.

Polling for an "OK" file: This approach is for ESB to wait for a 0 KB file with same name as data file's name but appended with a an extension such as ".OK" or ".complete". Assumption: The application and ONLY application knows when it has FINISHED writing a file to the disk.

Risk: If not already implemented in the application, the additional functionality of generating the OK file needs to be implemented. The ESB also in this case will only deal with the archival of the OK file. Hence, it is possible for confusions to arise when there are lots of data file present in the ESB "Inbound" folder. Remember, in our previous options we used to archive the data files in order for ESB to not pick the files in duplicate.

Conclusion: Despite minimum risks involved in this option, in my opinion this should along with Option 2, might be a preferred implementation. Due to non-transactional behavior of the file system, the basic fact is that only an application knows when it finishes writing the file. Hence it is important that it provides another app (ESB) a signal when it finishes writing the file and option 3 is completely based on this theory. These options also assume that the application does not have any other means to communicate with ESB apart from a File System.