I Hate Cubicle

What are inverse or ill-posed problems? Part 1.

2010-11-13T08:24:00.000-08:00

First publications on inverse and ill-posed problems appeared in first half of 20th century. They were related to scientific research in physics (inverse problem of quantum theory of scattering, electrodynamics and acoustics), geophysics (inverse problems of electric sounding, seismology, theory of potential), astronomy and other areas of natural sciences. With the emergence of computers, inverse problems penetrated all scientific disciplines that use mathematical methods. Major application are geophysics, astronomy, data visualization, medical and industrial tomography, defect detection and distance sounding and many others.

In the direct problem of mathematical physics, researchers tend to find (explicitly or approximately) the functions describing various physical phenomena, such as the propagation of sound, heat and seismic waves, electromagnetic waves, and so on. The properties of the medium (the equation coefficients), as well as the initial state of the process (nonstationary case) or its properties on the boundary (in the case of a limited area and / or in the stationary case) are assumed to be known. However, it is the properties of the medium, in practice, are often unknown. This means that you need to formulate and solve the inverse problem, which is required to determine any coefficients of the equations, or unknown initial or boundary conditions or the location, boundaries and other properties of the region in which the process under study. These objectives are in most cases ill-posed (i.e., at least one of the three properties of well-posedness - a condition of existence, uniqueness and stability of the solution with respect to small variations in the data of the problem, is violated). And the unknown coefficients of the equations are as a rule, the density, electrical conductivity, thermal conductivity and other important properties of the medium. Also, very often in inverse problems is required to find the location, shape and structure of the inclusions, defects, sources (heat, vibration, stress, pollution) and so on. It is no wonder that with such a broad set of applications, the theory of inverse and ill-posed problems since its inception has been one of the most rapidly developing areas of modern science.

Everyone solves inverse and ill-posed problem every minute. Humans solve them, as a rule, quickly and efficiently (if they are in good health and a clear mind). Let's consider, for example, visual perception. It is known that our vision fixes only a finite number of points around us. Then how are we seeing everything? Brain (in this situation - a powerful personal computer) uses seen points (interpolates and extrapolates) to recover all that the eye did not percept. It is clear that to recover true picture (in the general case - the volume and color) using only several points can only be the case when it is already more or less familiar with (most of the subjects and images we have seen, and sometimes touched by hands). That is, despite the strong ill-posedness (non-uniqueness and instability of the solution) of the problem (to restore the observed object and all that surrounds it by several points), the brain solves this problem pretty quickly. Why? It uses a wealth of experience (a priori information). And anyway, if we want to understand something complex enough to solve the problem, the probability of error which is large enough, we tend to arrive at an unstable (ill-posed) problem.

We can say that people (especially inclined to seek unconventional solutions) are constantly facing inverse and ill-posed problems. In fact, everyone understands how easy to make a mistake when trying to restore the past, based on some facts of the present (trace the motives and details of the crime on the available evidence, understand the reasons for initiation and development stages of the disease on the survey results and so on). Or look into the future (anticipated development of the child, the direction of the country and another complicated processes). Or penetrate into the zone of inaccessibility and to understand what is happening there (to investigate the internal human organs, to detect mineral deposits, to learn something new about the universe and so on). In fact, any attempt to expand the boundaries of direct (sensory, visual, auditory, etc.) perception of the world leads to an ill-posed problems. It would seem that we can say that, having learned to solve the stable (well posed) problem, mathematicians have moved on to more complex unstable (inverse and ill-posed) problems. But historically this is not true, since all the ages man has been surrounded by ill-posed problems, and mathematicians tried to solve such problems, dispensing with the relevant terms.

Inverse and ill-posed problems are united by one important feature - the instability of solutions to small errors in measurement data. In most interesting cases, inverse problems are ill-posed and such problems, as a rule, can be formulated as inverse with respect to certain direct (well posed) problems. But since inverse and ill-posed problems are historically formulated and studied quite often independently and in parallel, nowadays both terms are used in the scientific literature.

Summarizing, we can say that experts in the inverse and ill-posed problems
are involved in the study of properties and methods for the regularization of unstable problems. In other words, mathematicians are trying to create and explore sustainable methods of approximation of unstable maps. From the viewpoint of linear algebra this is a search of approximate methods for finding the normal pseudosolutions of systems of linear algebraic equations with rectangular, singular or poorly conditioned matrices. In functional analysis, a prime example of ill-posed problem is an operator equation Aq = f, where A is compact (completely continuous). Recently published studies interpret some problems of mathematical statistics as the inverse problem of probability theory. From the viewpoint of information theory, experts in the inverse and ill-posed problems investigate the properties of maps of compacts with a high epsilon-entropy in the table with a small epsilon-entropy.

In this introductory article we have learned about definitions of inverse and ill-posed problems, examples of their practical application and possible areas of development.

Next part will cover historical background of inverse and ill-posed problems.

Amazon AWS Micro Instances

2010-09-09T08:35:00.000-07:00

At last Amazon AWS came up with cheaper versions of their instances, as they call ‘micro instances’.

This change is pretty logical from economical standpoint, as in most cases clients need less resources than usually given by cloud provider. It is also important to note, that less usage implies less money spent on cloud resources. This was one of the reasons of my migration from Amazon AWS to Rackspace CloudServers product line, as it turned out to be cheaper and more powerful alternative.

Introduction of lower rate, however, still won’t make me move all of my deployments back to Amazon AWS, as besides economical part of the equation we still observe disparity in critical subsystems such as CPU cores. See below an output of cat /proc/cpuinfo ran on one of my Rackspace instances:

root@myhost:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.002
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
bogomips : 3994.49
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.002
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
bogomips : 3994.49
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 2
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.002
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
bogomips : 3994.49
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 3
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.002
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
bogomips : 3994.49
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

root@myhost:~#

Amazon AWS Micro Instance provides 613 MB of memory, up to 2 ECUs (for short periodic bursts), EBS storage only, 32-bit or 64-bit platform

It is easy to see that CPU-wise Amazon has to add at least 2 cores to be comparable. We should also take into account other critical subsystems like disk and network throughput. Rackspace declares 20Mbit throughput for their 512Mb instance, while I couldn’t find official numbers from Amazon. Perhaps this could be a good exercise for yourself during weekends.

UP-START Cloud Awards

2010-08-05T23:17:00.000-07:00

Cloudcor and PricewaterhouseCoopers announce the 2010 UP-START Cloud Computing Awards recognition event to be held November 15, 2010 on Day 1 of the UP 2010 Cloud Computing Conference.

Are you redefining the next big thing in Cloud Computing?

Is your company defining cloud technology or is your product going to change the future of the Information Communications Technology (ICT) landscape?

Cloudcor Inc, seeks to broaden the end-user community’s understanding of where and how Cloud Computing will influence our future life, within both public and private sector environments.

We are looking for disruptive and next generation Cloud Computing companies, as well as innovative stealth mode / emerging solution providers, to step up to the plate and showcase their capabilities!

Here is the opportunity for you to showcase your products and solutions at the UP START Cloud Computing Awards.

Distinguished delegates to the inaugural UP START Cloud Awards will comprise of:

- Cloud Industry Leaders,
- Venture Capitalists,
- Expert Industry analysts
- Tech / Cloud evangelists,
- Innovators in the Cloud Computing space.

The UP START Cloud Awards, provide the ideal platform for emerging Cloud providers; (or those newly entering the Cloud Space), providing a springboard to success in this rapidly shifting and fast paced environment.

Winners in each category will receive a coveted industry recognition, as well as a tangible prize; consisting of an all expenses paid “Gold Partner Status” sponsorship, for the forthcoming “world renowned” CloudSlam’11 conference in Early 2011. ($20,000+ equivalent value)

The judges comprising of recognized Cloud Computing industry experts, will award the most promising UP and coming business cases and unique creativity / differentiators, of the product / solution being profiled.

This will consist of the following award categories:

- Fastest Growing Cloud Computing Company Award
- Overall most innovative Cloud Computing Provider Award
- Most promising start-up cloud provider Award
- Efficiency in Technology Award

The judges on the day, will be announced shortly before the conference start date.

This ensures that the short listing process (and subsequent finalists), are determined solely on individual submission merit.

Members of the broader Cloud Computing, end-user and ICT community can also have their say by providing comments and feedback on finalists presentations.

If you have a viable Cloud Computing solution, driving innovation, or have the next big thing in Cloud Computing this is the event for you!

We invite stealth mode, emerging, first / second round financed, and more established companies, with innovative and cutting edge Cloud Computing solutions to get involved!

You can register for the UP START Cloud Awards by filling in an online form here.

The deadline for nomination submissions is September 30, 2010.

The finalists in each class will be selected from the submitted applications and will be invited to demonstrate in front of the judges and the conference audience on Monday,15th November 2010. (free access to all finalists)

Selection Process

Legal Disclaimer

Venue Information

Quick thoughts on Cloud Computing and Future of Video Games

2009-10-04T09:13:00.000-07:00

With the emergence of video games technology into online delivery model, it is easy to see that Chinese market tends to be the most promising spot due to its impressive growth rate (rising from $10 billion dollars in 1998 to $20 billion dollars currently) and huge market size (338 million Internet users). The success of online delivery model is explained by extremely relaxed approach to licensed software in China.

I think projects like OnLive , that allow you to play almost all titles without downloads or install through micro-console or computer, could make a fortune in China (after they conquer North America), once local infrastructure is ready to support broadband gaming.

This could also force existing players like Microsoft Xbox, Sony Playstation and Nintendo Wii to re-think their current strategies and possibly adapt existing hardware to support services like OnLive, so they could manage their market share.

New Online Conference Analyzes Top Cloud Computing Trends and Challenges

2009-02-11T07:15:00.000-08:00

The world’s leading experts and authorities in cloud computing to convene on April 20-24 at Cloud Slam ’09 to analyze and uncover top trends and challenges.

TORONTO – February 10, 2009 – Cloud Slam ’09, a virtual conference developed to promote collaborative analysis of the latest trends and challenges in the world of cloud computing, will commence on April 20. Organized by more than 30 leading experts and authorities in the field, this four-day conference will highlight research, developments and accomplishments by industry leaders whose contributions are helping to shape the future of cloud computing. Key topics will include the significant benefits of an open cloud, enterprise readiness of IaaS architectures, and the impact of cloud computing on application developers.

According to industry experts and analysts, cloud virtualization is expected to explode over the next 18 to 36 months, and by 2012 will account for new tech spending. Cloud Slam ’09 gives industry leaders and professionals keen insights into published research, unique and evolving ideas and best practices, as well as an opportunity to network with leading experts in cloud computing. The conference is designed to be a career-enhancing experience for researchers, engineers, directors and IT professionals. It’s also an ideal opportunity for corporate leaders to glean information on the latest innovations in cloud computing, generate leads and develop ideas on how to capitalize on what’s estimated to eventually become a $100 billion dollar market.

“Information sharing and collaboration will be key components in the evolution of cloud computing, especially now, as we prepare for its continued, industry-wide impact on both small businesses and large corporations,” said Khazret Sapenov, CTO of Cloudcor Inc. and conference organizing committee member. “Our conference brings together members from the world’s largest cloud computing community, comprised of more than 8,000 members, to share and analyze existing research, and tackle hot topics including infrastructure, implementation, privacy and compliance.”

Cloud Slam ’09 will feature several keynote speakers, and presentations from academia on the current and future aspects of cloud computing. Among the featured speakers are Songnian Zhou, CEO of Platform; Simon Crosby, CTO, Virtualization and Management, Citrix Systems, Inc.; Russ Daniels,VP and Chief Technology Officer, Cloud Services Strategy, HP; Hal Stern, SVP Global Systems Engineering; Maximilian Ahrens, CTO of Zimory; and Jayshree Ullal, President and Chief Executive Officer, Arista Networks.

Cloud Slam ’09 is presented by more than 30 industry and academic leaders. The organizing committee includes the following experts: Geoffrey Fox, Professor of Computer Science, University of Indiana; Greg Pfister, Independent Computer Hardware Professional, Distinguished Engineer at STG, IBM; Tim Mather, V.P. & Chief Security Strategist at RSA Conferences; Eliezer Dekel, Senior Technical Staff Member, Manager, Distributed Middleware IBM Research Laboratory in Haifa; Kristi Schultz, STSM - Blue Cloud Ensemble Management, IBM; Gabriel Sallah, HPC and Virtualisation Architect Global markets at Standard Chartered Bank in Singapore; John Savageau, CTO, CRG West; Achim Luhn, Director Innovation Projects at Siemens IT Solutions and Services; Khazret Sapenov, CTO, Cloudcor Inc.; Dave Graham, EMC Corporation; Paco Nathan, Adknowledge; Krishna Sankar, Distinguished Engineer at Cisco; Chris Sears, SunGard; Randy Bias, VP Technology Strategy, GoGrid; Chris Marino, CEO, SnapLogic; Andre Merzky, IT Analyst CCT, Louisiana State University; Igor Edelman, VP Engineering at Bank of America; J. Andrew Rogers, VP Geospatial Systems at Neopolitan; Subra Kumaraswamy, Sr. Security Manager at Sun Microsystems; Richard Donaldson, COO at United Layer; Brian Cinque, Data Center Architect at Sun Microsystems; Sassa NF, Technology Advisor; Eric Moore, Technical Consultant, HP Software; Rob Gingell, CTO and EVP Product Development for Cassatt; Daniel Schwartz, Banking Industry; and Edward Browdy, Information Assurance and Security Professional.

Conference registration is now open. The early bird registration fee is $40, and available until February 15. General admission is $52. For more information or to register, visit www.cloudslam09.com. For sponsorship opportunities, contact Khazret Sapenov at info@cloudslam.org.

About Cloud Slam ‘09

Cloud Slam ’09 is the world’s largest virtual cloud computing conference, covering technology, business models, industry implementation experiences, regulatory compliance, privacy and security, risk management and scientific research in the world of cloud computing. The conference will take place April 20-24. For more information, visit www.cloudslam09.com.

###

Alfresco Share Flash Preview

2009-01-05T08:16:00.000-08:00

During recent work with Alfresco Share, I encountered issue similar to one described in this topic http://forums.alfresco.com/en/viewtopic.php?f=9&t=14380

Apparently this was related to swf tools generating flash preview using parameter, supplied by Alfresco. My hack was to modify following config file /home/apache-tomcat-5.5.27/webapps/alfresco/WEB-INF/classes/alfresco/thumbnail-service-context.xml

to change default flash version from 9 to 10, since more and more clients upgrade to recent version, so this becomes more actual.

After change it should look like:

<bean class="org.alfresco.repo.thumbnail.ThumbnailDefinition">
<property name="name" value="webpreview" />
<property name="mimetype" value="application/x-shockwave-flash"/>
<property name="transformationOptions">
<bean class="org.alfresco.repo.content.transform.swf.SWFTransformationOptions">
<property name="flashVersion" value="10"/>
</bean>

Perhaps there should be more flexible way to detect version of Flash player and communicate it back to Alfresco/SWF tools to render proper code.

Cloud Computing Conference in 2009

2008-12-23T18:56:00.000-08:00

The Cloud Slam Conference is the world's premier cloud computing event, covering research, development, innovations and education in the world of cloud computing. The Technical Program is unmatched, and reflects the highest level of accomplishments in the cloud computing community, while the invited presentations feature an exceptional lineup of speakers. The panels, workshops, and tutorials are selected to cover a range of the hottest topics in cloud computing.

Descriptions of our conference tracks are presented below.

Technology
Data Centers, HPC, Cloud Storage, Hardware/Equipment, Software, Platforms, Virtualization, SaaS, E-government (eScience and eEducation), Security, Monitoring, Distributed Technologies, Data in Cloud, CDN

Industry Implementation Experience
Learn about experiences transitioning traditional IT resources into the cloud, efforts to use the cloud to support business processes that have been traditionally supported by IT. Usage/ industry focus such as application to and lessons learned from use in institutional and / or retail financial businesses, life sciences, manufacturing, retail, medicine, pharmaceutical, etc. Discussion of practical usage and issues discovered/addressed could help in promoting a shift in perception of Cloud Computing.

Business Models
This track discusses how to make money in the cloud - for startups, incumbents and Venture Capital/Investments. How to make a career in Cloud Computing(recommended for for HR/recruiters/candidates/institutions/courses).

Legal Aspects: Compliance, Privacy
What is personal information (the definition is quite broad, but varies from jurisdiction to jurisdiction)? What is privacy? (a.k.a. "information self-determination" i.e. the right or ability of indivudals to exercise a measure of control over the collection, use and disclsoure of their personal information by others). Where is the personal data stored (profound implications for jurisdiction and applicable laws, transparency, accountability, recourse, etc.).
What laws apply to the data in question (privacy, security, business transaction, consumer prrotection, etc). Is the data secure? (How would you know? What assurances are available? Does assurance level depend on sensitivity? Is it auditable?).
Is individual consent provided? How? is it informed? Can it be revoked? Can conditions be attached?).
Who owns the data? (In the U.S., it is exclusively the organziation's, everywhere else, ownership is a "shared responsibility")
Who (e.g. third parties, agents) is the data shared with? Under what circumstances?
Are there data breach disclosure requirements in place - if a breach occurs will you be informed? What remedies, if any, will be offered?
What steps or measures can people and organizations to limit exposure and lability?
In this Information era of unlimited storage, mirrors, backups, etc - has data deletion (when no longer needed for its original purpose) become an obsolete idea?

Research

Presentations from academia on the current and future aspects of cloud computing(e.g. running MATLAB/Star-P/Mathematica parallel calculations for life sciences projects in cloud environments or storing data sets such as the Human Genome, U.S. Census and labor statistics in cloud to make the information easier to access for researchers).
We invite you to propose a presentation for Cloud Slam 2009. The deadline for abstract submissions is Monday, January 19, 2009. Click here for details on submission guidelines.

For more information see website at http://cloudslam09.com

LinkedIn Event

Facebook Event

Integration of workflow, rules and monitoring in compute cloud

2008-09-23T18:59:00.000-07:00

This is what I have planned to describe long ago, but didn't due to many reasons. While working with provisioning workflows in virtualized environments (we had only Xen in 2005), I have seen a need to separate workflows and rules, so it is easy to maintain business logic.

Take a look at sequence diagram below:

It describes transaction from capturing event from sensor network (in this case gmond metrics) and executing arbitrary code in response to specified condition.

Ocean floor data center

2008-09-17T20:46:00.000-07:00

During discussion of floating data centers at Cloud Computing Group,

I've come to an idea of placing data centers in proximity to trans-ocean cable landings on ocean floor. Crazy, huh ? :)

It might be started at the border of neutral waters, around 5-10 km from shoreline and later expanded along major submarine cable systems.

Perhaps, initially it might be non-maintainable sealed containers with racks, powered by either

a) elements using natural forces like waves or streams

b) galvanic elements

c) hydrogen (there's definitely no lack of water)

and cooled by external medium (also water).

ps. Original images are from this really cool site.

Update on November 14, 2010.

Here's another technology to get electric power for DC.

Swedish company Minesto's underwater kite resembles a child's toy as it swoops and dives in ocean currents. But since seawater is 800 times as dense as air, the small turbine attached to the kite — which is tethered to the ocean floor — can generate 800 times more energy than if it were in the sky. Minesto calls the technology Deep Green and says it can generate 500 kilowatts of power even in calm waters; the design could increase the market for tidal power by 80%, the company says. The first scale model will be unveiled next year off the coast of Northern Ireland.

Patterns in Cloud Computing

2008-06-02T20:58:00.000-07:00

Recently I posted a small piece of my thoughts on cloud computing in mailing list ( http://groups.google.ca/group/cloud-computing/browse_thread/thread/86ee2f8f4afd71c1# ) on patterns in cloud computing, comparing on demand computing resource pools to human resources.
Indeed, both target same goal: descrease fixed costs of ownership and have more space for manoeuvre. Though it has some differences: computing resources (CPU, RAM, disk storage etc) are of known quality, while people (consultants) not always.

Another thing, that was lurking in my mind, is that if you compare compute cloud to an Earth subsystem like hydrosphere (which is actually fed by precipitation from atmosphere, read clouds), then it has similar properties like streams, pollution, draining, freezing etc.

Freezing of storage resources might be compared to loss of elasticity, like inability of some storage space to efficiently store optimal amount of data. For example 1Gb of storage has 10 copies of same file (or slightly different versions) under different name. I also see some dynamic process eating RAM, which reminds me of water changing physical state (steam or geyser, if you want).

This might be solved by process of deduplication, so storage unit gets back its original properties (defragmentation comes to mind as well).

I don't know yet, whether these restoration/optimization processes should belong to system (compute cloud solution) or to the entities, that run inside cloud, future will show.

Jumping aside, I would also note a popular Observer pattern, used in publish/subscribe system, that I suggested to use for cloud implementations in some other posts to the same discussion list, for the purpose of reducing bandwidth/resources consumption.

I'll get you informed further in this thread. If you think, that I'm crazy - comments are welcome.

Alfresco Cluster in Compute Cloud (Amazon EC2)

2008-05-27T21:11:00.001-07:00

Synopsis: This article describes simplified process of setting up Alfresco Cluster in Cloud Computing environment (Amazon Elastic Compute Cloud).

Pre-requisites.

You need to have Amazon Elastic Compute Cloud account (make sure you have files like pk-ABCDAABCDAABCDAABCDAABCDAABCDAABCDA.pem and cert-ABCDAABCDAABCDAABCDAABCDAABCDAABCDA.pem )
Basic knowledge of Alfresco CMS
Some knowledge of Amazon EC2 tools

Part 1. Introduction

What is Alfresco and why it is cool.

Alfresco is the leading open source alternative for enterprise content management. The open source model allows Alfresco to use best-of-breed open source technologies and contributions from the open source community to get higher quality software produced more quickly at much lower cost.

The Benefits of Using Alfresco

Ease-of-Use
Intelligent Virtual File System – As simple to use as a shared drive through CIFS, WebDAV or FTP
Google®-Like Search and Yahoo!®-Like Folder Browsing

Developer Productivity

Aspect Oriented Rules Development through Simple-to-Use Wizards
Rules and Actions Managed in the Server once for all Interfaces

Best-Practice Collaboration

Pre-Configured Smart-Space Templates – Project Structure, Content, Logic, Lifecycles
Forums – Threaded Discussions on Folders or Documents

Administrator Productivity

Simple Server Install and No Client Install
Advanced Content Security Management
Advanced Search/Knowledge Management
Sophisticated Content, Attribute, Location, Object Type and Multiple Taxonomy/Category

Distributed Architecture
Highly Scalable and Fault Tolerant Service Oriented Architecture

Open Source

Dramatically Lower Cost

What is Amazon EC2 and why it is also cool.

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
Amazon EC2's simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon's proven computing environment.

Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient applications and isolate themselves from common failure scenarios.

Why a combination of both is uber-cool.

Benefits of the cloud allow you to provision preconfigured Digital Assets Repository on demand and scale it dynamically, thus reducing total ownership costs considerably.

Part 2. Getting your Cluster in 15 minutes.

Run management instance.

I use ElasticFox extension to do basic tasks on EC2. Run instance of Amazon Machine Image ami-68af4b01. After state of the instance changes from pending to running, you need to get Public DNS and put it into browser address bar. This should bring Web User Interface for Alfresco Cluster Management. You can login with following credentials: username alfresco, password alfresco. Now you should have left navigation with options:

* Add New Alfresco Cluster

* Alfresco Cluster Home

* Cluster Provisioning Status

* General Settings

* List All Alfresco Clusters

* Provision Alfresco Cluster Nodes

Setup EC2 credentials.

Using client like Putty or other ssh client, log in to Public DNS address and download your key files to dedicated path (e.g. /root). Now you need to edit following file /var/www/sites/all/modules/alfresco_ec2/ws-amazon-ec2-client.php and add names of your key files.

Provision minimal cluster.

At this point you are ready to provision your minimal 3-node Alfresco Cluster. Proceed to web interface and choose Add New Alfresco Cluster option. Fill in the form and choose List All Alfresco Clusters option in the left navigation to see your new cluster. To actually provision nodes to it, click on radio button for specific cluster and press submit button 'Provision Nodes for Selected Cluster Now'. This will lock browser for pretty long time (around 6-7 minutes). I have modified version of software, that handles this in a more elegant fashion. After it finishes provisioning transaction, you'll see results on the screen.

Test cluster functionality.

See ElasticFox extension and note new instances, that are respectively, DB Node, Master Node and Slave Node. Get its Public DNS addresses and open it in browser (it should like http://ec2-00-100-200-100.compute-1.amazonaws.com:8080/alfresco). Log in to one of the instances (or re-log in) as admin:admin and Add Content. Then log in to another instance and see if content of the space shows new file.

Conclusions.

This setup allows one click provisioning of Alfresco Cluster on EC2 and provides enterprise class digital assets managment system, that can be used for multiple purposes.

Future Work.

Scale-On-Demand.

High-Availability Configuration.

Media Streaming Solutions.

LDAP integration.

Single Sign On/Out.

Contact.

If you have any questions, please write me an email at sapenov at gmail dot com.

Cloud Computing

2008-04-09T19:38:00.000-07:00

Cloud Computing Group in LinkedIn has been created.
If you are involved in areas of parallel calculations, high performance computing and related fields or just interested in future directions of subject, please join the group at http://www.linkedin.com/e/gis/61513/6213F13BB1AA

or write to join.cloud.computing@gmail.com

For discusssion join mailing list at http://groups.google.ca/group/cloud-computing

Archive of posts http://computingondemand.blogspot.com/

Setting Up Windows 2008 in Virtual Box

2008-02-13T19:52:00.000-08:00

Synopsis: This article describes how to setup networking and activation in Microsoft Windows 2008 Server Std running under VitualBox.

If you have installed Windows 2008 Server and can't pass activation step, this howto is for you.

Algorithm - short description:

1. Run explorer.
2. Get VirtualBox Guest Additions installed.
3. Install Network Driver from VirtualBox cdrom disk and reboot.
4. Activate Windows.

Detailed description:

For step 1 read and use this article http://www.petri.co.il/bypass-windows-server-2008-activation.htm

Step 2 is fairly easy - just click 'Install ...' option in dropdown of VirtualBox and inside guest os accept autorun option.

Step 3 is as easy as updating driver and pointing to CDROM drive with VirtualBox guest additions directory called something like amd_net

Step 4 might require reboot and then just follow screen instructions.

At this point you should have Windows 2008 server running under VirtualBox.

Benchmarking CouchDB - READ 6000

2008-02-01T21:02:00.000-08:00

[root@domU-12-31-35-00-3D-82 ~]# ab -t100 -n 6000 -c 6000 http://ec2-67-202-22-9.compute-1.amazonaws.com:5984/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking ec2-67-202-22-9.compute-1.amazonaws.com (be patient)
Completed 600 requests
Completed 1200 requests
Completed 1800 requests
Completed 2400 requests
Completed 3000 requests
Completed 3600 requests
Completed 4200 requests
Completed 4800 requests
Completed 5400 requests
Finished 6000 requests

Server Software: inets/develop
Server Hostname: ec2-67-202-22-9.compute-1.amazonaws.com
Server Port: 5984

Document Path: /
Document Length: 215 bytes

Concurrency Level: 6000
Time taken for tests: 9.224306 seconds
Complete requests: 6000
Failed requests: 182
(Connect: 0, Length: 182, Exceptions: 0)
Write errors: 0
Non-2xx responses: 5870
Total transferred: 2145510 bytes
HTML transferred: 1269980 bytes
Requests per second: 650.46 [#/sec] (mean)
Time per request: 9224.306 [ms] (mean)
Time per request: 1.537 [ms] (mean, across all concurrent requests)
Transfer rate: 227.12 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 4 509 1078.4 10 7714
Processing: 8 1118 2672.2 69 8908
Waiting: 4 1003 2562.0 34 8896
Total: 12 1627 2844.8 97 9208

Percentage of the requests served within a certain time (ms)
50% 97
66% 453
75% 3028
80% 3103
90% 8319
95% 8571
98% 8966
99% 9039
100% 9208 (longest request)
[root@domU-12-31-35-00-3D-82 ~]#

Benchmarking CouchDB - READ 2000

2008-02-01T20:43:00.000-08:00

[root@domU-12-31-35-00-3D-82 ~]# ab -k -t100 -n 3000 -c 2200 http://ec2-67-202-22-9.compute-1.amazonaws.com:5984/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking ec2-67-202-22-9.compute-1.amazonaws.com (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Finished 3000 requests

Server Software: inets/develop
Server Hostname: ec2-67-202-22-9.compute-1.amazonaws.com
Server Port: 5984

Document Path: /
Document Length: 215 bytes

Concurrency Level: 2200
Time taken for tests: 2.830302 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Non-2xx responses: 3009
Keep-Alive requests: 0
Total transferred: 1073998 bytes
HTML transferred: 646720 bytes
Requests per second: 1059.96 [#/sec] (mean)
Time per request: 2075.555 [ms] (mean)
Time per request: 0.943 [ms] (mean, across all concurrent requests)
Transfer rate: 370.28 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 11 26.9 2 115
Processing: 3 12 22.6 5 168
Waiting: 1 9 21.8 3 165
Total: 5 23 41.6 7 280

Percentage of the requests served within a certain time (ms)
50% 7
66% 9
75% 9
80% 36
90% 68
95% 126
98% 188
99% 203
100% 280 (longest request)
[root@domU-12-31-35-00-3D-82 ~]#

Benchmarking CouchDB - READ

2008-02-01T07:11:00.000-08:00

etch:~# ab -k -n 210 -c 140 http://ec2-67-202-22-9.compute-1.amazonaws.com:5984/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking ec2-67-202-22-9.compute-1.amazonaws.com (be patient)
Completed 100 requests
Completed 200 requests
Finished 210 requests

Server Software: inets/develop
Server Hostname: ec2-67-202-22-9.compute-1.amazonaws.com
Server Port: 5984

Document Path: /
Document Length: 44 bytes

Concurrency Level: 140
Time taken for tests: 0.412173 seconds
Complete requests: 210
Failed requests: 52
(Connect: 0, Length: 52, Exceptions: 0)
Write errors: 0
Non-2xx responses: 52
Keep-Alive requests: 0
Total transferred: 60434 bytes
HTML transferred: 18132 bytes
Requests per second: 509.49 [#/sec] (mean)
Time per request: 274.782 [ms] (mean)
Time per request: 1.963 [ms] (mean, across all concurrent requests)
Transfer rate: 143.14 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 71 53.7 110 118
Processing: 0 130 92.1 142 261
Waiting: 0 129 92.3 141 260
Total: 0 202 126.9 252 377

Percentage of the requests served within a certain time (ms)
50% 252
66% 257
75% 264
80% 270
90% 369
95% 373
98% 376
99% 376
100% 377 (longest request)
etch:~# ab -k -n 210 -c 130 http://ec2-67-202-22-9.compute-1.amazonaws.com:5984/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking ec2-67-202-22-9.compute-1.amazonaws.com (be patient)
Completed 100 requests
Completed 200 requests
Finished 210 requests

Server Software: inets/develop
Server Hostname: ec2-67-202-22-9.compute-1.amazonaws.com
Server Port: 5984

Document Path: /
Document Length: 44 bytes

Concurrency Level: 130
Time taken for tests: 0.538343 seconds
Complete requests: 210
Failed requests: 0
Write errors: 0
Keep-Alive requests: 0
Total transferred: 68900 bytes
HTML transferred: 11440 bytes
Requests per second: 390.09 [#/sec] (mean)
Time per request: 333.260 [ms] (mean)
Time per request: 2.564 [ms] (mean, across all concurrent requests)
Transfer rate: 124.46 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 22 130 70.0 134 257
Processing: 27 139 62.5 138 261
Waiting: 2 128 67.3 124 257
Total: 240 269 18.4 283 284

Percentage of the requests served within a certain time (ms)
50% 283
66% 283
75% 283
80% 284
90% 284
95% 284
98% 284
99% 284
100% 284 (longest request)
etch:~#

Benchmarking CouchDB

2008-01-24T13:47:00.000-08:00

After installation of CouchDB, I was wondering about it's performance, so I made some benchmarking, and it looks pretty good to me - 2,000 concurrent requests.

I only had to raise number of open files per process by using

echo "204854" > /proc/sys/fs/file-max

ulimit -n 200000

to get rid of this message
OS error code 24: Too many open files

I'll publish results as Google docs, so they are posted to this blog automatically and I'll provide just a link to results.

This are read tests :

http://ihatecubicle.blogspot.com/2008/02/benchmarking-couchdb-read.html

http://ihatecubicle.blogspot.com/2008/02/benchmarking-couchdb-read-2000.html

http://ihatecubicle.blogspot.com/2008/02/benchmarking-couchdb-read-6000.html

I'm working on creation of files with sample data to emulate document for insertion, so we can test write efficiency.

Amazon EC2 - server response 400 (Bad Request)

2008-01-23T15:21:00.000-08:00

etch:/mnt# ec2-upload-bundle -b khaz_debian_elasticdrive041_02 -m /mnt/image.manifest.xml -a key -s secretkey

Setting bucket ACL to allow EC2 read access ...
Error: could not create or access bucket khaz_debian_elasticdrive041_02: server response 400 (Bad Request)
ec2-upload-bundle failed

trying --debug option showed, that create bucket operation fails.

Searching Amazon EC2 forums didn't give anything valuable, so I'd tried to create bucket manually and voila - it spits error that sound like "You've tried to create more buckets than allowed". So deleting some garbage buckets helped to get rid of this hard to trace error (at least for me).

Setup Elastic Drive in Jeos/VMWare

2008-01-23T11:03:00.000-08:00

Sometimes if you try to run Elastic Drive

root@khazJeosED:/opt/e# sudo elasticdrive /etc/elasticdrive.ini -d

and get following error:

Traceback (most recent call last): File "/usr/bin/elasticdrive", line 5, in pkg_resources.run_script('ElasticDrive==0.4.1-FREE-5-r54', 'elasticdrive') File "build/bdist.linux-i686/egg/pkg_resources.py", line 448, in run_script File "build/bdist.linux-i686/egg/pkg_resources.py", line 1173, in run_script File "/usr/bin/elasticdrive", line 175, in
File "/usr/bin/elasticdrive", line 156, in main
File "build/bdist.linux-i686/egg/elasticdrive/app_server.py", line 99, in start_drives File "build/bdist.linux-i686/egg/elasticdrive/s3_simple.py", line 57, in __init__ File "/usr/lib/python2.5/site-packages/boto-0.9d-py2.5.egg/boto/s3/connection.py", line 103, in create_bucket raise S3ResponseError(response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden RequestTimeTooSkewedThe difference between the request time and the current time is too large.2008-01-23T18:47:06Z439AD07AEA9A14C6900000a7RNB5bY7n1muIce2Ht40zcMUj4qkwJV6vtHdgWFzA6ezn3aov171Ov1UrHXWweZWed, 23 Jan 2008 01:31:49 GMT Exception exceptions.AttributeError: "s3_simple_engine instance has no attribute 'locker'" in > ignored

it means that local time should be adjusted to comply Amazon S3 requirement (max difference might be 10 minutes).

Extract time part from response 2008-01-23T18:47:06Z minus 5 hours, and use it to set time in sync. Run following command:

date --set="2008-01-23 13:47:43"

run again

root@khazJeosED:/opt/e# sudo elasticdrive /etc/elasticdrive.ini -d

if no errors, lets check it all goes well

#ps aux
...
root 5480 0.2 1.0 18176 5284 ? Ssl 13:59 0:00 /usr/bin/python /usr/bin/elasticdrive /etc/elasticdrive.ini -d
root 5482 0.2 1.0 18128 5348 ? Ssl 13:59 0:00 /usr/bin/python /usr/bin/elasticdrive /etc/elasticdrive.ini -d
root 5485 0.0 0.9 9660 4836 ? S 13:59 0:00 /usr/bin/python /usr/bin/elasticdrive /etc/elasticdrive.ini -d
root 5488 0.0 0.1 2620 1000 pts/0 R+ 14:00 0:00 ps aux

let's mount filesystem

root@khazJeosED:/opt/e# sudo mount -o loop /fuse2/ed0 /s3
mount: you must specify the filesystem type

this might mean that you need to run

#mke2fs -b 4096 /fuse2/ed0

mke2fs 1.40.2 (12-Jul-2007)
/fuse2/ed0 is not a block special device.Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
65536 inodes, 65536 blocks
3276 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=67108864
2 block groups
32768 blocks per group, 32768 fragments per group
32768 inodes per group
Superblock backups stored on blocks:
32768
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 33 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.
root@khazJeosED:/opt/e#

now we can run the previous command:

root@khazJeosED:/opt/e# sudo mount -o loop /fuse2/ed0 /s3
root@khazJeosED:/opt/e#

Let check it went fine:

root@khazJeosED:/opt/e# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 2.8G 474M 2.2G 18% /
varrun 252M 44K 252M 1% /var/run
varlock 252M 0 252M 0% /var/lock
udev 252M 44K 252M 1% /dev
devshm 252M 0 252M 0% /dev/shm
/fuse2/ed0 248M 144K 236M 1% /s3

Running Selenium Grid on Amazon EC2

2008-01-21T08:25:00.000-08:00

Synopsis:
This article describes a process of setting up and running Selenium Grid web testing tool in distributed fashion.

Pre-requisistes:
You should have an Amazon Elastic Compute Cloud account
It is implied, that you have basic Unix/Linux administration skills and some experience with Amazon EC2 tools
Some knowledge of Selenium testing ( read more at http://www.openqa.org/selenium-grid/ )

How to start rolling:
1. Run instance of AMI ami-fd37d294 and get Public DNS of the instance.

2. Login into ssh terminal, using Public DNS.

3. Run following commands:

#cd /usr/share/selenium-grid-0.9.3
#ant sanity-check

Instances of this AMI can be run as "hub" or "remote control" depending on your layout. If this is going to be a remote control:

#ant -Dport= -Dhost= -DhubURL= launch-remote-control
ant -Dport=5555 -Dhost=testhost -DhubURL=http://your-hub-url:4444 -Denvironment="Firefox on Linux" launch-remote-control

Future work:
- pass parameters before launch, so it runs in hub or rm mode, all neat and automated
- add Firefox on Windows and Safari on OS X profiles

Geotourism in Kazakhstan

2008-01-20T12:00:00.000-08:00

What is geotourism ?
National Geographic defines geotourism as tourism that sustains or enhances the geographical character of a place -its environment, culture, aesthetics, heritage, and the well-being of its residents. Geotourism incorporates the concept of sustainable tourism -that destinations should remain unspoiled for future generations- while allowing for enhancement that protects the character of the locale. Geotourism also adopts a principle from its cousin, ecotourism -that tourism revenue can promote conservation- and extends that principle beyond nature travel to encompass culture and history as well: all distinctive assets of a place.

Challenge
Nominate your favorite example of geotourism -- defined by National Geographic as tourism that sustains or enhances the geographical character of a place: its environment, heritage, culture, aesthetics, and the well-being of its residents.
All nominees will receive a personal email inviting them to enter the National Geographic-Ashoka's-Changemakers Geotourism Challenge launching on jan 30, 2008. All nominators who tell their first-person story become eligible to win one of two whl.travel prizes.
http://www.changemakers.net/en-us/competition/geotourism

My nomination - Falcon hunt in Central Kazakhstan

Hunting with golden eagles in Kazakhstan, is ancient and deeply esteemed tradition, and hunters-berkutchi are one of the most esteemed persons in a society.
The first world falconry festival took place on July14-15, 2007 in Reading, Berkshire, United Kingdom http://www.falconryfestival.com/

Festival gathered teams from 33 countries, including USA, France, Germany, Belgium, Japan, Brazil, China, South Korea, Turkmenistan, England and others. Kazakhstan team was recognized the best one.

If you are new to falconry, Alan Gates has good article about hunting with falcon at http://www.avmv20.dsl.pipex.com/Articles/The%20Hunt.htm

I would like to present my friend's falconry estate at Central Kazakhstan, that offers falcons, hawks and golden eagles to hunt wolf, fox, hare, black-cocks, ducks, partridges and pigeons. Since 1995 they hold international competitions “Salburn” - hunting with birds and Kazakh “tazy”- wolfhound, while riding horses.

For more details visit their website at http://www.pmicro.kz/~falcon/falcon_e/
or just give them a call:
Tileukabyl Esembekuly: 7 7212 45-41-02 , 7 300 244 65 69, tleu.e@rambler.ru
Karlygash Makataeva: 7 7272 71-26-17, 7 300 755 20 86, kmakataeva@yahoo.com

[UPDATE]

Found good article on the topic in Chicago Tribune http://www.chicagotribune.com/news/nationworld/chi-falconry_webfeb03,1,2451718.story

Benchmarking Windows 2003 Server in Amazon EC2

2008-01-10T19:45:00.000-08:00

Abstract: Some notes on trying to run Windows 2003 Server EE on Amazon EC2 extralarge type of instances (4 way).

SMP
I tried to use Qemu's -smp n feature on Amazon Elastic Compute Cloud, but it still uses only one processor. It is possible to set affinity and run each Qemu instance on individual processor, but this is very nontypical scenario. Also this article explains Qemu's "rdtsc" usage on SMP hosts http://lists.gnu.org/archive/html/qemu-devel/2007-03/msg00652.html

ramfs
Moving system disk to RAM decreased disk-related IO by 10-15%, but CPU-bound bottleneck still slowed system down.

ACPI SMP
Windows Device Manager showed that our installation uses MP kernel according to "ACPI Multiprocessor PC", ACPI APIC MP HAL (Halmacpi.dll).

I'll keep you informed on my other findings.

How to setup LDAP authentication in Alfresco

2008-01-07T08:17:00.000-08:00

Synopsis: This articles describes a process of setting up an LDAP authentication in Afresco content management system.

OpenLDAP
Setting it up is pretty trivial, I used yum. It is important to add initial entries into fresh installation, since it comes totally empty and spits errors. This is how I did it:
#nano khaz.ldif
dn: dc=khaz-domain,dc=com
objectClass: domain
dc: khaz-domain

#slapadd -l khaz.ldif
#chown ldap:ldap /var/lib/ldap/objectClass.bdb
#/etc/rc.d/init.d/ldap restart

Atlassian Crowd
Crowd is a web-based single sign-on (SSO) tool that simplifies application provisioning and identity management. I used it as front-end tool for OpenLDAP to manage users.
Install the software and login into administration panel at something like http://yoursite.com:8095/crowd/console
choosing Directories tab and click on add directory. Choose "Crowd supports several connectors such as Active Directory, Sun ONE and Open Directory. " Connector button and fill in details (of OpenLDAP installation).

Alfresco
I used bundled version (tomcat + alfresco) with HSQL database, which might be switched to another one like MySQL.
Firstly, I tried it on my desktop in VMware server and then on Amazon EC2 instance, running under Fedora Core 6.
During initial stage I turned on debug mode to see exactly what was going on, and it really helped me to trace LDAP communication messages between my OpenLDAP server and Alfresco.
Use this settings as a guidance:
/opt/alfresco/tomcat/shared/classes/alfresco/extension/chaining-authentication-context.xml
/opt/alfresco/tomcat/shared/classes/alfresco/extension/ldap-authentication-context.xml

http://s3.amazonaws.com/khaz_download/chaining-authentication-context.xml

http://s3.amazonaws.com/khaz_download/ldap-authentication-context.xml

Adding users to Alfresco
Log in to Crowd panel.
Choose Principals tab > then OpenLDAPForAlfresco (this is how I named it, yours might have different name) in Directory dropdown and hit Search button
This should bring a list of users in directory. To add new user, locate Add Principal in Principal Browser tab and click on it. This will change to form, where you fill in user details and select proper directory for user to belong to.
Upon successful creation of user account, you can test it in Alfresco at http://youralfrescoinstallation.com:8080/alfresco At this point all users are managed outside of alfresco and might be easily attached to other services like single sign-on and OpenID.

How to access Alfresco from Vista using Network Location

2007-11-29T09:30:00.000-08:00

Abstract: This article describes setup process of Network Location in Microsoft Windows Vista to access Alfresco.

Alfresco is open source digital assets management system and has lots of useful features like support of multiple protocols for remote access. One of them is access to repository via CIFS/WebDAV, or, in plain English, from your Desktop. This gives you an easy way to transfer bunch of files to digital repo, preserving folder hierarchy.

Here is a list of actions:

1. Go to My Computer, right-click on it and choose 'Add Network Location'.
You'll see wizard window - click Next button, on the next screen choose custom location and click Next button, then in dialog window, which should ask for location of your website, type in something like http://youralfrescoinstallation.com:8080/alfresco/webdav and hit Next button, this will bring popup windows that propmts for access credentials, add username and password.

2. If all is good, you will see next window, that asks 'what do you want to name this location' - type in the name you want and hit Next button. At this step you should see final screen of the wizard, that confirms successful creation of network location, leave all default choices and hit Finish button.

3. Now you have web folder linked to your Alfresco digital assets repository and can drag
and drop files in or copy files to your computer.

Enjoy your alfresco ! :)

Possible problems:
You might get stuck on step one, because of differences in protocol implementation, when whatever you enter doesn't let you proceed to next step.
I have resolved this problem by downloading and installing following update for Microsoft Windows Vista Home Edition (called Software Update for Web Folders (KB907306) ).

http://www.microsoft.com/downloads/details.aspx?FamilyId=17C36612-632E-4C04-9382-987622ED1D64&displaylang=en

Integrating Java and C apps on Linux.

2007-11-15T14:54:00.000-08:00

Abstract: This article consists of several parts that describe my experience with integrating C programs into Java-based infrastructure like Java Messaging.

Part 1. How to build C client for JMS on Fedora Core 6

In this part I'll show how to build C client for Java Message Queue on Fedora Core 6. The story began with some cool application in C with needed functionality, while I had no time and resources to re-write it in Java.We'll need to install following packages:

-bash-3.1# yum install nss.i386 compat-libstdc++-33.i386 screen

This will install C compatibilty libraries, NSS and dependent libraries.
Now we need to get Open Message Queue server and client code (community version).These commands will download and extract distro into /opt/sun/mq directory.

-bash-3.1#wget --no-check-certificate https://mq.dev.java.net/files/documents/5002/66518/mq4_1-binary-Linux_X86-20070816.jar
-bash-3.1#mkdir -p /opt/sun
-bash-3.1#cd /opt/sun
-bash-3.1#unzip mq4_1-binary-Linux_X86-20070816.jar
-bash-3.1#cd mq

Lets run messaging server by issuing following commands:

-bash-3.1#screen

This will create a screen session.

-bash-3.1#/opt/sun/mq/bin/imbrokerd -tty

This will start the server outputting a bunch of information into terminal.

To detach from screen session press Ctrl + A + D (so your server will be still running and you'll be able to return to it later by hitting screen -r)

In this tutorial I would limit it to single example, while you have three available in the directory. Let's use producer_consumer for the sake of simplicity and just change working directory:

-bash-3.1#cd demo/C/producer_consumer

There you'll see two C files - Consumer.c and Producer.c

At this point you are ready to start building your clients. Let's issue following commands:

-bash-3.1# g++ -DLINUX -D_REENTRANT -I/opt/sun/mq/include -o Producer -L/opt/sun/mq/lib -lmqcrt

You may see warning that complaints on possible conflict:

Producer.c/usr/bin/ld: warning: libstdc++.so.5, needed by /opt/sun/mq/lib/libmqcrt.so, may conflict with libstdc++.so.6

If we look at directory contents now, it should contain binary with file name Producer. If you try running it, it might spit out following error:

-bash-3.1# ./Producer

./Producer: error while loading shared libraries: libmqcrt.so.1: cannot open shared object file: No such file or directory

This is easily fixed by adding libraries location to the path. Lets create file

-bash-3.1# nano /etc/ld.so.conf.d/mq.conf

and add /opt/sun/mq/lib to it.

Then just run following command:

-bash-3.1#ldconfig

Now you should be able to build both source files and run it without errors:

-bash-3.1#g++ -DLINUX -D_REENTRANT -I/opt/sun/mq/include -o Consumer -L/opt/sun/mq/lib -lmqcrt Consumer.c

-bash-3.1#g++ -DLINUX -D_REENTRANT -I/opt/sun/mq/include -o Producer -L/opt/sun/mq/lib -lmqcrt Producer.c

-bash-3.1#ln -s Producer p; ln -s Consumer c; ls -al

At this point you should have binaries of C client that are able to publish and read messages to Messaging server.

For usage options run

-bash-3.1#./Producer -help

-bash-3.1#./Consumer help

If you need to know more about messaging server, that we used in this tutorial, visit this url http://www.sun.com/software/products/message_queue/index.xml. In the next part of tutorial I'll describe integration of C client into existing application. Stay tuned.

Why programmers hate cubicle (office space)

2007-10-20T21:48:00.000-07:00

Interesting discussion on the topic (Do cubicles hurt productivity?) http://www.computerworld.com/blogs/node/1510

How to setup Elastic Drive in Fedora Core 6 running under VMWare

2007-10-01T09:50:00.000-07:00

Abstract: This article describes instructions on installation of Elastic Drive in FC6 that runs under VMWare.

Having received several requests on helping to install Elastic Drive under Fedora Core 6, I've set up a short tutorial, that you'll see below.

VMWARE

To start, you'll need to have VMware Player or Server installed.
I've downloaded Fedora Core from this place , but looks like it not very different from the fedora site in a sense that you still need to set some configuration for your Fedora. This would take about 2 minutes to accomplish.

Set your system time to be in sync with S3, since it doesn't allow difference more than 10 minutes. I would go to System > Administration > Date and Time and checkmark 'system clock uses UTC' and choose Americas etc to have EDT. Then set you system time to match S3 precisely in terminal (or you may do it with above utilities):

date -s "enter S3 time here"

Then I've applied latest updates and proceeded to terminal, where entered following:

SETUP PRE-REQUISITES

yum install fuse fuse-devel python-devel gcc-c++ gcc gcc-devel glibc-devel glibc-headers libgomp libstdc++-devel

SETUP ELASTIC DRIVE

cd /opt

wget http://www.elasticdrive.com/uploads/media/elasticdrive-0.4.0_dist.tar.gz

tar zxf elasticdrive-0.4.0_dist.tar.gz

ln -s elasticdrive-0.4.0_dist e

cd e

./install

mkdir -p /fuse /fuse2 /data/s3

RUN ELASTIC DRIVE

At this point, if you don't have any error, you have elasticdrive application installed and ready to run. As recommended, you'll need to edit configuration file, located at /etc/elasticdrive.ini (we'll omit configuration details, since it is well described at the site)
To run the application, enter following in terminal:

elasticdrive /etc/elasticdrive.ini -d

ps aux | grep elastic

ls -al /fuse2

If you see it found in list of processes, then it is running. Also the latter command should show something like ed0.

SETUP FILESYSTEM

If you have all of the above show up, let's do file system stuff:

mke2fs -b 4096 /fuse2/ed0
// choose 'y' when you see Proceed anyway? (y,n)

mount -o loop /fuse2/ed0 /data/s3

TEST ELASTIC DRIVE

At this point you should have file system ready for work. Try copy some files into /data/s3 or test it some other ways.

df -h

cd /data/s3

wget http://www.elasticdrive.com/uploads/media/elasticdrive-0.4.0_dist.tar.gz

How to use Elastic Drive VMWare virtual appliance

2007-09-25T11:19:00.000-07:00

Abstract: This article describes installation of Elastic Drive application as VMWare virtual appliance.

VMWARE PLAYER OR SERVER

- download virtual appliance from following link:
http://s3.amazonaws.com/vmware_appliances/khaz_VMWare_Debian_ElasticDrive_Public.rar
- unpack it to your Virtual Machines directory.
- File > Open > Browse > path to your dir > filename.vmx
- in Debian terminal, when you see command prompt, use following credentials: username khaz / password khaz / superuser khazrocks99

VMWARE APPLIANCE CONFIGURATION

- change root passwd, delete user khaz
- setup environment up to your taste, if you want

CONFIGURING APPLICATION

- edit /etc/elasticdrive.ini
- add credentials in [drives] section (after 's3://S3ACCESSKEY:S3SECRETKEY')
- create bucket name (maybe instance id + number)
- specify disk size by setting '&blocks=' (default setting of 65536 blocks gives you 268Mb)

RUNNING APPLICATION

- /etc/init.d/elasticdrive_khaz start
- application should automatically startup on reboot

USING YOUR NEW FILESYSTEM

Now you can use your filesystem, that is persistent to Amazon Simple Storage (S3).
Try copy some files to /data/s3, or 'umount /fuse2/ed0' (if you have lots of data, this would require more time, well, up to 30 minutes), 'ls -al /data/s3' should show nothing when unmounted, reboot, 'ls -al /data/s3'.

EVACUATION OF DATA FROM VMWARE APPLIANCE

This basically covers unmounting your current filesystem and transferring it's state to external persistence source.
Initially I tested creation of 24Gb, placing different size files and unmounting. First umount took about 20 minutes or so, all subsequent worked in a less than minute.

Setting Up Elastic Drive at Amazon EC2

2007-09-20T14:08:00.000-07:00

Abstract: This article contains intructions on running Debian Linux (etch) on Amazon EC2 with Elastic Drive that links instance to S3 storage.

EC2 CONTROL TOOLS
- run instance of following AMI ami-7cfd1815, get public DNS address

- ssh in to that public DNS address, using following credentials: username khaz / password khaz / superuser khazrocks99

EC2 INSTANCE CONFIGURATION
- change root passwd, delete user khaz
- setup environment up to your taste, if you want

CONFIGURING APPLICATION
- edit /etc/elasticdrive.ini
- add credentials in [drives] section (after 's3://S3ACCESSKEY:S3SECRETKEY')
- create bucket name (maybe instance id + number)
- specify disk size by setting '&blocks=' (default setting of 65536 blocks gives you 268Mb)

RUNNING APPLICATION

- /etc/init.d/elasticdrive_khaz start
- application should automatically startup on reboot

USING YOUR NEW FILESYSTEM

Now you can use your filesystem, that is persistent to Amazon Simple Storage (S3).
Try copy some files to /data/s3, or 'umount /fuse2/ed0' (if you have lots of data, this would require more time, well, up to 30 minutes), 'ls -al /data/s3' should show nothing when unmounted, reboot, 'ls -al /data/s3'.

EVACUATION OF DATA FROM EC2 INSTANCE

This basically covers unmounting your current filesystem and transferring it's state to external persistence source. Initially I tested creation of 24Gb, placing different size files and unmounting. First umount took about 20 minutes or so, all subsequent worked in a less than minute.

Treemap Widget in Dojo GFX

2007-08-23T21:55:00.000-07:00

Last week I was working on implementing 'bin packing' algorithm and visualizing output on 2D plane, using DOJO toolkit. At the time of this post it's available here: http://facebook.enomalylabs.com/treemap7.php

It shows categories of footprint and descendant items. Implemented in dojo gfx (SVG/VML).

Some information on bin packing:

In computational complexity theory, the bin packing problem is a combinatorial NP-hard problem. In it, objects of different volumes must be packed into a finite number of bins of capacity V in a way that minimizes the number of bins used.

There are many variations of this problem, such as 2D packing, linear packing, packing by weight, packing by cost, and so on. They have many applications, such as filling up containers, loading trucks with weight capacity, and creating file backup in removable media.

Since it is NP-hard, the most efficient known algorithms use heuristics to accomplish results which, though very good in most cases, may not be the optimal solution. For example, the first fit algorithm provides a fast but often nonoptimal solution, involving placing each item into the first bin in which it will fit. It requires O(n log n) time. The algorithm can be made much more effective by first sorting the list of elements into decreasing order (sometimes known as the first-fit decreasing algorithm), although this does not guarantee an optimal solution, and for longer lists may increase the running time of the algorithm.

update:

Got an idea to use the treemap widget in Ganglia project.
It contains two rationales:
a) use treemap to visualize metrics, so it is more compact and doesn't poll every X amount of time.
b) use client-side code to render readings and be more compact way to display multiple parameters

In the following demo I used dojo toolkit (version 0.9) and php:http://facebook.enomalylabs.com/ganglia.php

Dojo's SVG support is still experimental though (i think it is a wrapper interface for both SVG implementations).

I'm planning on implementation of another approach which uses curves instead of rectangles (based on Voronoi tesselations).

How to install Attansic L2 Network Driver in Fedora Core 7

2007-08-05T21:17:00.000-07:00

Having bought new box for my kids, I've got ASUS P5GC-MX motherboard with Intel chipset and installed Fedora Core 7 (Live CD).

All worked fine except integrated network card from Attansic (Taiwan). First of all, I couldn't figure out what type of device I have, there were two Linux drivers provided: L1 and L2.

A little bit of googling and voila - it's L2 !
As it happened, the driver required building Linux kernel module (responsible for network) as LiveCD install didn't have it by default.
What ?!
Building kernel module ? Hmm... (why don't I just install windows, I asked myself several times during this procedure :) )

My default method to install all the required components is yum, but it was useless without network setup. So I had to download all packages to my another desktop, copy them to USB key and then use the key at target Linux box.

You would need to get kernel-headers-2.6.21-1.3194.fc7.i386.rpm, kernel-devel-2.6.21-1.3194.fc7.i686.rpm, gcc with dependencies
- glibc-2.6-3.i386.rpm,
- glibc-headers-2.6-3.i386.rpm,
- libgomp-4.1.2-12.i386.rpm,
- glibc-devel-2.6-3.i386.rpm,
- gcc-4.1.2-12.i386.rpm,
- cpp-4.1.2-12.i386.rpm,

and install it using RPM (e.g. rpm -Uvh _name_here.rpm).

To verify that you are ready to build kernel module, check content of /usr/src path, it should contain 'kernels' directory.
Now follow instructions and change current dir to /src/ dir with driver source code and do:
'make install'.

I had some issues with drivers provided on motherboard CD, as happened it was outdated version something like 0.2.40.0. It complained on missing config.h etc

After some googling, I figure out, that version 0.2.40.0 is for older kernels (before 2.6.21). Then I found newer version of the driver and got rid of config.h and other header errors, but got another ones about non-declared members of
struct. A short look at C code showed that some debugging parts are excessive, so simple commenting did the trick.

At this point I'd ran 'make install' , then 'cd /lib/modules//kernel/drivers/net', 'insmod atl2.ko' and got network device working. Then I went to Network configuration and added ip address and dns information.
After device activation I was able to browse internet.

Newer version of driver is (was) available here:
http://launchpadlibrarian.net/7382416/L2-linux-driver_new.rar

RPMs are available at ftp://download.fedora.redhat.com/pub/fedora/linux/releases/7/Everything/i386/os/Fedora/

Good luck !

ps. also found this site (it has Linux Drivers package, but download was so slow that i didn't test the package) http://support.asus.com/download/download_item.aspx?model=P5GC-MX&product=1&type=map&mapindex=3&SLanguage=en-us

pps.

When applying updates that include kernel updates, after reboot it loads new kernel and module we built is not loaded, so no network would be available. In my case I just edit grub.conf and set old kernel (the one we build driver for) and reboot.

Technology advancement

2007-06-13T12:54:00.000-07:00

interesting how computer technology advanced last decade, in 1990 it was a privilege of math or EE graduates and some kids, who had techically inclined parents and now my 5 year old son is happily playing pretty complex games.

Alfresco 2.0 Community Edition

2007-04-06T08:18:00.000-07:00

Today I've installed Alfresco 2.0 community edition. The process wasn't different from version 1.4, usually I use .war file and deploy it on tomcat. This time it was done at amazon elastic compute cloud instance under fedora core 4, using my previous image with tomcat and mysql already setup. To cleanup, i've removed .war file and respective directories, dropped mysql database and recreated it again. [to be continued]

Тест на знание русского языка

2007-04-02T14:35:00.000-07:00

Мой результат "Теста от ульяновского губернатора"

110 из 116

Пройти тест на грамотность

Cluster Management

2007-03-18T11:19:00.000-07:00

Today I've installed:
Moab Cluster Suite^_® is a professional cluster management solution that integrates scheduling, managing, monitoring and reporting of cluster workloads. Moab Cluster Suite simplifies and unifies management across one or multiple hardware, operating system, storage, network, license and resource manager environments to increase the ROI of cluster investments. Its task-oriented graphical management and flexible policy capabilities provide an intelligent management layer that guarantees service levels, speeds job processing and easily accommodates additional resources.

http://domu-12-31-34-00-01-eb.usma2.compute.amazonaws.com:8080/map/

and Gold
Gold is an open source accounting system developed
by Pacific Northwest National Laboratory (PNNL) as part of the Department of Energy (DOE) Scalable Systems Software Project (SSS). It tracks resource usage on High Performance Computers and acts much like a bank, establishing accounts in order to pre-allocate user and project resource usage over specific nodes and timeframe. Gold provides balance and usage feedback to users, managers, and system administrators.

Users of Moab Workload Manager can integrate with Gold to track and charge job resource usage, providing greater control over who is using the cluster or grid resources.

http://domu-12-31-34-00-01-eb.usma2.compute.amazonaws.com/cgi-bin/gold/index.cgi

Overall resume: pretty decent piece of software

EC2 Amazon - QEMU Windows Images

2007-03-18T11:10:00.000-07:00

This is an update. I have my account approved at sourceforge, so all qemu images will be hosted at this address (http://sourceforge.net/projects/qemuwinrepo/).

EC2 Amazon - QEMU Windows Images

2007-02-28T10:45:00.000-08:00

Recently I've succeeded to run Microsoft Windows Server 2003 on Amazon Elastic Compute Cloud. That was done by running it under Qemu (read more on it). I've used trial version of windows, to be safe from legal point of view (though I'm not a lawyer) and ran installation procedure, which led to working copy of OS, available for remote administration. It was released for public
as freely available Amazon Machine Image (read more on it).

Some people were asking to create AMI image with MSSQL server or Oracle etc, but I've got an idea to create a repository of Qemu images (an analog of VMware appliances), so that you can have base Qemu image and different overlays for various install layouts. This would save space and traffic, as well as decrease installation time considerably.

Currently I'm waiting on response to my registration at SourceForge. If they allow hosting of this repository, I'll put my images there and place a link to it here.

Zend PHP IDE review

2007-02-28T09:39:00.000-08:00

Though I use EditPlus on a daily basis, here is list of features in Zend IDE, that I personally like:

Templates
Conditional breakpoints
Docking windows
Profiler (it also has remote profiling)
Code Analyzer
Clone View
CVS Integration
Real-time errors
CVS Diff
Goto source
Code folding

I like Google more and more

2007-02-28T08:15:00.000-08:00

I like recent Google's Apps like spreadsheet and docs, now I don't need to have MS/Open Office installed. It also has integration of your mail, blog, calendar and documents, which is very convinient. Good work Google !