Understanding the relationship between SNAT and connection limitations in full proxy intermediaries.
If you’ve previously delved into the world of SNAT (which is becoming increasingly important in large-scale implementations, such as those in the service provider world) you remember that SNAT essentially provides an IP address from which a full-proxy intermediary can communicate with server-side resources and maintain control over the return routing path.
There is an interesting relationship between intermediaries that leverage two separate TCP stacks (such as full-proxies) and SNAT in terms of concurrent (open) connections that can be supported by any given “virtual” server (or virtual IP address, as they are often referred to in the industry). The number of ephemeral ports that can be used by any client IP address is 65535. Programmer types will recognize that as a natural limitation imposed by the use of an unsigned short integer (16 bits) in many programming languages.
Now, what that means is that for each SNAT address assigned to a virtual IP address, a theoretical total of 65535 connections can be open at any other single address at any given time. This is because in a full-proxy architecture the intermediary is acting as a client and while servers use well-known ports for communication, clients do not. They use ephemeral (temporary) ports, the value of which is communicated to the server in the source port field in the request. Each additional SNAT address available increases the total number of connections by some portion of that space. As you should never use ephemeral ports in the privileged range (port numbers under 1024 are traditionally reserved for firewall and other sanity checkers - see /etc/services on any Unix box) that number can be as many as 64512 available ports between the SNAT address and any other IP address. For example, if a server pool (virtual or iron) has 24 members and assuming the SNAT address is configured to use ephemeral ports in the range of 1024-65535, then a single SNAT address results in a total of 24 x 48k = 1,152k concurrent connections to the pool. If the SNAT is assigned to a virtual server that is targeting a single address (like another virtual server or another intermediate device) then the total connections is 1 x 48k = 48k connections.
Obviously this has a rather profound impact on scalability and capacity planning. If you only have one SNAT address available and you need the capabilities of a full-proxy (such as payload inspection inbound and out) you can only support a limited number of connections (and by extension, users). Some solutions provide the means by which these limitations can be mitigated, such as the ability to configure a SNAT pool (a set of dedicated IP addresses) from which SNAT addresses can be automatically pulled and used to automatically increase the number of available ephemeral ports.
Running out of ephemeral ports is known as “ephemeral port exhaustion” as you have exhausted the ports available from which a connection to the server resource can be made. In practice the number of ephemeral ports available for any given IP address can be limited by operating system implementations and is always much lower than the 65535 available per IP address. For example, the IANA official suggestion is that ephemeral ports use 49152 through 65535, which means a limitation of 16383 open connections per address. Any full-proxy intermediary that has adopted this suggestion would necessarily require more SNAT addresses to scale an application to more concurrent connections.
One of the advantages of a solution implementing a custom TCP/IP stack, then, is that they can ignore the suggestion on ephemeral port assignment typically imposed at the operating system or underlying software layer and increase the range to the full 65535 if desired. Another major advantage is making aggressive use of TIME-WAIT recycling. Normal TCP stacks hold on to the ephemeral port for seconds to minutes after a connection closes. This leads to odd bursting behavior. With proper use of TCP timestamps you can recycle that ephemeral port almost immediately.
Regardless, it is an important relationship to remember, especially if it appears that the Load balancer (intermediary) is suddenly the bottleneck when demand increases. It may be that you don’t have enough IP addresses and thus ports available to handle the load.
WILS: Write It Like Seth. Seth Godin always gets his point across with brevity and wit. WILS is an ATTEMPT TO BE concise about application delivery TOPICS AND just get straight to the point. NO DILLY DALLYING AROUND.
The social enterprise space continues to gain acceptance. After years of promise, the market seems to be truly hitting its stride. This is best represented by the continued funding of social enterprise services such as Gist, which today announced it raised another $4 million.
A catalyst for this new energy has to be how it's working together with cloud computing, which is fueling apps and ecosystems that provide a fertile place for SaaS services to grow and build their subscription base. Demand is increasing for these services as many enterprises continue to find ways to drive efficiency and optimize returns. The way to do that is through cloud-based services that leverage Web-oriented architectures and offer affordable subscription-based services.
Gist represents this trend. The Seattle-based company offers a contact management system that organizes contacts across platforms such as email and social networks and lets you see that information using a dashboard environment.
Sponsor
Gist competes with services such as Batchbook and Etacts. We mentioned these companies yesterday in a post about the crossover between collaboration and CRM services. All of these companies are part of the Google Apps Marketplace, which is serving as a community for apps to gain a foothold in the market. We've heard off the record that the company is using Capsule CRM, a service with a focus on project collaboration. It's unconfirmed, but it shows that Google Apps customers are taking advantage of the marketplace.
Gist also has a partnership with IBM Lotus Notes, again showing how SaaS providers can leverage the strength of established heavyweights.
More proof comes from Atlassian , which recently raised $60 million form Accel Partners. The Google Apps Marketplace is becoming one of its most important channels for selling its services.
Jive Software is a bit different. it just raised $30 million and is moving toward an IPO. Jive is competing more directly with Google and Microsoft, though the company integrates with Sharepoint and is also in the Google Apps Marketplace. Jive is seeking to build its own apps marketplace. It is launching an open API in the Fall.
The $4 million round for Gist came from Paul Allen's Vulcan Capital and the Foundry Group out of Boulder Colorado. Gist previously received $6.75 million in May of last year.
All of these companies have the flexibility of leveraging the vast infrastructure that has been built for the purpose of serving applications. The cloud is the catalyst and will continue to serve as fertile ground for social technologies in the enterprise.
Discuss
The benefits of scalability and flexibility touted by cloud computing are rapidly pushing its adoption forward. And while there are still hesitations in some sectors about the emerging technologies - questions about security, control, and interoperability - the majority of tech experts see moving to the cloud as inevitable.
Sponsor
Nevertheless, it's still fairly common to hear the argument that the public cloud isn't quite ready or quite right for the enterprise. But a recent Forrester report contends that enterprise might not be ready for the private cloud either. Pointing to a survey that found only about 5% of IT shops have the experience internally to make the move to the cloud, the report, authored by James Staten with Christian Kane, Robert Whiteley, contends that many companies simply aren't ready.
Calling virtualization "the yellow brick road to the cloud," Staten recommends companies assess their own technological maturity and their adoption of virtualization in order to ascertain whether private cloud environments are the right move right now.
According to Staten, you're ready for the private cloud if you meet the following criteria:
Staten notes that preparing your IT operations for Stage 4 will take many years, but says that moving to the cloud doesn't have to wait until then. Rather than a whole-scale switch, he recommends moving smaller projects and investments, such as development and testing, into the cloud. He also recommends outsourcing the internal cloud to an Infrastructure-as-a-Service cloud provider - preferably one that will offer some training.
The "reality check" that the Forrester report offers might not be what some IT departments or cloud advocates want to hear. But as Staten concludes, "The economics of cloud computing is too compelling for you sit on the sidelines waiting for the hype to die down. You need to start investing in IaaS now to understand how best to leverage it. You also need to embrace the fact that your developers and line-of-business leads aren't waiting for you to figure this out."
Discuss
by noreply@blogger.com (adriancole) at July 28, 2010 11:04 PM
by kvjacksn@gmail.com (Kevin L. Jackson) at July 28, 2010 04:09 PM
Russell Sinclair described how to link Microsoft Access 2010 tables to a SQL Azure database in his Access 2010 and SQL Azure post to the Microsoft Access blog of 6/7/2010. Ryan didn’t include screen captures in his narrative for creating the ODBC connection with the SQL Server Native Client 10.0 driver. I also encountered a problem that required a workaround to get links to a Northwind database working. This step-by-step tutorial adds screen captures and describes the workaround for primary key column name conflicts. It also describes an issue with adding subdatasheets to linked table datasheets.
Update 7/28/2010: Changed author name from Ryan McMinn to Russell Sinclair, added more details about and a workaround for SQL Azure’s lack of support for extended properties (required for Subdatasheets and Lookup fields), and added demo of pass-through queries.
Note: This tutorial assumes that you have a SQL Azure account with at least one populated database and have SQL Server Management Studio [Express] 2008 R2 available to edit primary key names of the linked tables.
Creating a SQL Azure Data Source Name (DSN)
1. Launch Access 2010, click the External Data tab and click the ODBC Database button to open the ODBC Select Data Source dialog:
2. Click the New button to open the Select a Type of Data Source dialog and select the System Data Source option:
3. Click Next to open the Select a Driver for Which You Want to Set up a Data Source dialog and select the SQL Server Native Client 10.0 driver:
4. Click Next and Finish to open the SQL Server Data Source Wizard’s first dialog, type a name and description for the data source, and type the full server name, including the tcp: prefix (tcp:servername.database.windows.net):
You’ll find it easier to copy the server name from SSMS’s Connect to Server dialog.
5. Click Next to open the second Wizard dialog, select SQL Server Authentication, type your administrative account name with an @servername suffix, and password, and mark the Connect to SQL Server check box:
6. Click Next to open the third Wizard dialog, change the default database from master to the Northwind[DS] database and accept the default ANSI options:
7. Click Next and Finish to open the ODBC Microsoft SQL Server Setup dialog and, optionally, click the Test Data Source button:
You tested the connection when you selected the Northwind[DS] database from the SQL Azure-generated list in step 6.
8. Click OK to save your Machine Data Source and return to the initial Select Data Source dialog with your new data source selected:
9. Click OK to open the SQL Server Login dialog, accept your Login ID and type your administrative password:
Options let you change your password, database, default language, application name, and workstation ID.
10. Click OK to open the Link Tables dialog. Click to select the tables to link to your Access front end:
SQL Azure Labs’ SQL Azure Data Sync application created the SQL Azure tables from the Northwnd.mdf database included in the Northwind and pubs Sample Databases for SQL Server 2000. For more information about using SQL Azure Data Sync, see:
Only those tables present in the original Access 97 Northwind database are marked for linking. SQL Azure Data Sync generates the dbo.TableName_tracking tables.
11. Click OK to generate the linked tables. If you marked the Save Password checkbox in the Link Tables dialog, you receive this warning message for each table:
Working Around Invalid Primary Key Names
Using the Northwnd.mdf database, you receive the following message about an invalid Primary Key PK_dbo.TableName for each table you attempt to link:
12. To eliminate these errors, which prevent linking the tables, open the SQL Azure database in SQL Server Management Studio 2008 R2 and change the name of each primary key to PK_TableName as shown here for the Categories table:
Tip: When changing the name of PK_dbo.Order Details, remove the space (PK_OrderDetails) for safety. If you don’t you might encounter issues elsewhere.
Editing a Linked SQL Azure Table in a Datasheet
13. Repeat steps 9, 10 and 11 to create links and open one of the tables for editing:
14. Add a new item to the table to verify updatability:
15. Close and reopen the linked table to verify the record’s addition:
16. Open and scroll the Orders and Order Details tables to compare performance with tables linked from an on-premises and SQL Azure Northwind database. I find performance with a 350-kBps (inbound) DSL connection to compare reasonably well with the same database running on premises in SQL Server 2008 R2 Express.
Adding a Relationships Diagram and Subdatasheets
17. Optionally, create an Access database diagram in the Relationships window:
You can’t enforce referential integrity locally and Subdatsheet Name = [Auto] doesn’t generate subdatasheets for table datasheet views automatically:
You can Specify a top-level Subdatasheet Name, such as Table.dbo_Orders for the dbo_Customers table, which will automatically set Orders.CustomerID and Customers.CustomerID as the linked fields from the relationship specified in the diagram (as shown above).
18. Click the Datasheet button to open the Orders for the specified customer, Alfreds Futterkiste for this example:
The subdatasheet property values don’t persist because SQL Azure doesn’t support SQL Server 2008’s extended properties, which store the subdatasheet’s table and field names. Therefore, you lose the subdatasheet when you close and reopen the table. Lack of support for extended properties appears to make creating Lookup columns impossible.
19. The only obvious method of persisting subdatasheets is to create a SELECT * FROM tablename; query over each table and specify Query.queryname as the Subdatasheet Name property value:
This approach enables cascading subdatasheets, as shown here:
I’ve posted a message about this issue in the Access for Developers forum to determine if there’s a simpler workaround.
Pass-Through Queries
As Russ mentions in his blog post, you can execute pass-through queries against SQL Azure with the above DSN:
20. To create a pass-through query to verify SQL Azure’s lack of support for extended properties by returning an error when you attempt to execute the SQL Server 2008 fn_listextendedproperty system function, click the Create tab and the Queries group’s Query Design button, close the Add Table dialog, and click the Pass-Through button to open the SQL editing window and the query Property Sheet.
21. Select the ODBC Connect Str[ing] property’s builder button to open the Select Data Source dialog, click the Machine Data Source tab and select the SQLAzureNorthwindDS system data source you created in steps 1 through 10.
22. Click OK to open the SQL Server Login dialog, type your password, and click OK to connect.
23. Click Yes in the Connection String Builder dialog (refer to step 9) if you want to save the password in the connection string, where it can be seen by anyone who has authorization to enter query design mode. Otherwise click No (recommended).
24. Copy and paste the following example statement from SQL Server 2008 Books Online’s fn_listextendedproperty (Transact-SQL) topic:
SELECT objtype, objname, name, value
FROM fn_listextendedproperty(default, default, default, default, default, default, default);
to the SQL Window:
The full connection string, with security-sensitive values masked is:
ODBC;DSN=SQLAzureNorthwindDS;Description=Data Source for NorthwindDS on SQL Azure;UID=RogerJ@jcXXXXXXzaf; PWD=XXXXXX;DATABASE=NorthwindDS;
25. When you click Run to execute the query, you receive the following error message:
26. Test the query with a valid T-SQL statement that returns rows. The recordset will be read-only, as are all recordsets returned by pass-through queries and stored procedures.
Why Not Use an OLE DB Connection?
Stack Overflow user Troy asked Why doesn’t Microsoft support OLE DB connections to SQL Azure? in this 6/21/2010 question:
At the MSDN website it says, "Connecting to SQL Azure by using OLE DB is not supported."
There are other places on the web where folks report that it works fine for them after tweaking the server name in the connection string, such as here and here. Even SQL Server's Analysis Services uses OLE DB to connect to SQL Azure!
I develop a native/unmanaged application in Delphi that connects to SQL Server using ADO through the OLE DB provider for SQL Server. I'm considering adding SQL Azure support. It would be really helpful if I could reuse the majority of my code without not too much change. I probably wouldn't consider going this direction otherwise.
It would be helpful if Microsoft were more clear on why "OLE DB is not supported". If there are certain limitations within the use of OLE DB, what are they? Maybe I can work around them, or maybe it wouldn't affect me.
Microsoft also mentions that ODBC is supported. So could I use the "OLE DB provider to ODBC" and connect this way? Or is any combination that includes OLE DB "not supported"?
Ryan Dunn (@dunnry) from the SQL Azure team replied on the same date:
You can use it, however it has not been thoroughly tested for all cases. Essentially, it should work for most things, but there might be a few edge cases where it won't work. Until we document those cases, it remains unsupported. That being said, if you were to use and run into errors, we would love to know about it and prioritize that to be fixed.
by --rj (roger_jennings@compuserve.com) at July 28, 2010 01:38 PM
At what point does a CRM application become a collaboration service? And when does a collaboration service become a CRM application?
These are question we ask ourselves when we see the range of services with features that combine these two related application environments.
For instance, PBWorks today unveiled a service that allows people to collaborate with customers from the first point of communication all the way through the sale. PBWorks is known as an enterprise collaboration service. But with its new Customer Relationship Edition, PBWorks is now a more direct competitor against CRM companies like Salesforce.com.
Sponsor
You could ask the same question about Google Apps Marketplace, which gives Google Apps users the capability to turn an enterprise email environment inside out and use it as a contact management system with collaboration features.
Salesforce.com has developed Force.com into an ecosytem that integrates different applications into Salesforce.com. This in turn changes the definition for the overall service. It is not a pure CRM environment - it is a hybrid with a focus on CRM.
The Google Enterprise blog today illustrated how deep the integration has become between Google Apps and CRM applications. It divides CRM applications into three categories. The companies listed below link to their vendor pages in Google Apps Marketplace:
PBWorks' entrance into the CRM market means the company has to focus deeply on providing a core value as it does not have its own application platform. The company seems to be placing that focus on reaching out externally, not to an internal team as is often the case with CRM environments.
That approach represents where the market is heading. Collaboration tools have value, but to tie into the customer relationship cycle will give those services the potential to reach deeper into the enterprise. CRM services increasingly use collaboration services to extend the ways that customers can be brought into the communications process.
Does the CRM and collaboration combination point to Social CRM as the natural outcome for where the market is heading? The Google Apps Marketplace and Force.com ecosystems point to that possibility as does news from companies like PBWorks.
Discuss
Cloud computing has been a boon to tech startups, allowing them to build, launch and scale without substantial up-front investment in hardware. But at what point does the moving from the cloud to a data center make more sense - for both performance and cost?
Facebook announced plans earlier this year to build a custom data center in Prineville, Oregon, and Twitter announced last week that it plans to build one near Salt Lake City, Utah. And web app maker 37Signals isn't building its own data center, but it did reveal last week that it will move its infrastructure from Rackspace hosting to a colocation space in a Chicago data center.
Sponsor
As Jonathan Heiliger, Facebook's VP of Technical Operations said at last month's Structure 2010 conference, "For a consumer web site starting today, I would absolutely run on the cloud. It allows you to focus on building your product. But if you have 10 million users, that's a pretty big check I'm writing to someone else. How much control do I have?"
37Signals Operations Manager Mark Imbriaco explains the growth of 37Signals as such: since moving to the cloud four years ago, "we've grown from around 15 physical machines to a mixture of around 150 physical and virtual machines. We've also grown from having less than 1TB of data to on the order of 80TB of data today. Our needs have evolved a great deal as we've grown and we reached the point where it made sense for us to acquire our own hardware and manage our own datacenter infrastructure. The amount of flexibility that we have with our own environment makes it much easier for us to use some specialized equipment that meets our needs better than the solutions that Rackspace generally supports."
The desire for greater flexibility and control was also given by Twitter as rationale for its new data center. According to Twitter's Engineering Blog, "Twitter will have full control over network and systems configuration, with a much larger footprint in a building designed specifically around our unique power and cooling needs. Twitter will be able to define and manage to a finer grained SLA on the service as we are managing and monitoring at all layers. The data center will house a mixed-vendor environment for servers running open source OS and applications."
It isn't surprising that rapidly growing companies like Twitter and Facebook have reached a point where it pencils out to have their own data center. But what are the implications for cloud computing - something that promises infinite scalability?
And more importantly perhaps, the justifications given by Twitter and 37Signals for building a data center or moving to a colocation facility are less about cost and performance than they are about sufficient controls. Are public cloud providers doing enough to offer their enterprise customers with the flexibility and control they want?
Discuss
We wrote last week about the state of the cloud services market and how crowded it is becoming.
Cloud Bzz lists more than 30 providers. That includes companies that provide enterprise private clouds, public clouds and public sector clouds. John Treadway's take is that over the next few years there will be at most five or six companies that matter in the cloud infrastructure-as-a-service (IaaS) space.
Do you agree? Is the cloud services market too crowded?
Sponsor
But a shakeout may not happen for quite some time. If the market is really as large as some think, then it's logical that many of the companies in the space will survive. GoGrid, for instance, says the cloud infrastructure market is valued at $150 billion.
And study after study shows double digit increases in related cloud markets such as SaaS.
So, is there too much blood in the water? Is it too early to tell? What do you think?
Discuss
Bottles, birds, and packets: how the message is exchanged is less important than what the message is as long as it gets there.
Unpossible! How could the world not care about APIs? After all, it is APIs that make the Web (2.0) go around. It is APIs that drive the automation of infrastructure from static toward dynamic. It is APIs that drive self-service and thin-provisioning of compute and storage in the cloud. It is APIs that make cross-environment integration of SaaS possible. In general, without APIs we’d be very unconnected, un-integrated, un-collaborative, and in many cases, uninformed.
Now, it could be said that the world doesn’t care about APIs until they’re highly adopted, but unlike the chicken and the egg question (which may very well have been answered, in case you weren’t paying attention), it is still questionable whether the success of sites like Facebook and Twitter and the continued growth of SaaS darlings like Salesforce.com are dependent upon exactly that: their API.
The API is the new CLI in the network. The API is the web’s version of EAI (Enterprise Application Integration), without which we wouldn’t have interesting interactions between our favorite sites and applications. The API is the cloud’s version of the ATM (Automated Teller Machine) through which services are provisioned with just a few keystrokes and a valid credit card. The API is the means by which interoperability of cloud computing will be enabled because to do otherwise is to create the mother-of-all hub-and-spoke integration points. As Jen Harvey 
( co-founder of Voxilate, developer of voice-related mobile applications) pointed out, an API makes it possible to develop a user-interface that essentially obscures the underlying implementation. It’s the user-interface the users care about, and if you don’t have to change it as your application takes advantage of different clouds or services or technologies, you ensure that productivity and user-adoption – two frequently cited negative impacts of changes to applications in any organization – are not impacted at all. It’s a game-changer, to be sure.
So the API is the world in technology today, how could we not care about it?
Face it, if interoperability between anything were simply about the API then we’d have already solved this puppy and put it to sleep. Permanently. But it’s not about the API per se, it is, as William Vambenepe is wont to say, “it’s the model, stupid” [Edited per William's comment below, 7-27-2010]
Unfortunately, when most people stand up and cheer “the API” what they’re really cheering is “the model.” They aren’t making the distinction between the interface and the data (or meta-data) exchanged. It’s what's inside the message that enables interoperability because it is through the message, the model, that we are able to exchange the information necessary to do whatever it is the API call is supposed to do. Without a meaningful, shared model the API is really not all that important.
The API is how, not what, and unfortunately even if everyone agreed on how, we’d still have to worry about what and that, as anyone who has every worked with EAI systems can tell you, is the really, really, super hard part of integration. And it is integration that we’re really looking for when we talk about cloud and interoperability or portability or mobility, because what we want is to be able to share data (configuration, architectures, virtual machines, hypervisors, applications) across multiple programmatic systems in a meaningful way.
APIs, at least those on the web and taking advantage of HTTP, are little more than URIs. Doesn’t matter if it’s REST or SOAP, the end-point is still just a URI. The URI is often somewhat self-descriptive and in the case of true REST (which doesn’t really exist) it would be nearly completely self-documenting but it’s still just a URI. That means it is nearly a trivial exercise to map “/start/myresource” to “/myresource/start”. But when the data, the model, is expressed as the payload of that API call, then things get … ugly. Is one using JSON? Or is it XML? Is that XML OVF? Schemaless? Bob’s Homegrown Format? Does it use common descriptors? Is a load balancer in cloud a described as an application delivery controller in cloud b? Is the description of a filter required in cloud a using iptables semantics or some obscure format the developer made up on the fly because it made sense to her?
Mapping the data, the model, isn’t a trivial exercise. In fact without a common semantic model it would require not the traditional one-chicken sacrifice but probably a whole flock in order to get it working, and you’d essentially be locked-in for all the same reasons you end up locked-in today: it costs too much and takes too much effort to change. When pundits and experts talk about commoditization of cloud computing, and they do often, it’s not an attempt to minimize the importance of the model, of the infrastructure, but rather it’s a necessary step toward providing services in a consistent manner across implementations; across clouds. By defining core cloud computing services in a consistent manner and describing them in similar terms, advanced services can then be added atop those services in a like manner without impacting negatively the ability to migrate between implementations. If the underlying model is consistent, commoditized if you will, this process becomes much easier for everyone involved.
Consider HTTP headers. There is a common set, a standard set of headers used to describe core functions and capabilities. They have a common model and use consistent semantics through which the name-value pairs are described. Then there’s custom headers; headers that follow the same model but which are peculiar to the service being invoked. In a cloud model these are the differentiated value-added cloud services (VACS) Randy Bias mentions in his post regarding the announcement of OpenStack and the ensuing cries of “it will be the standard! it will save the world!”. The most important aspect of custom HTTP headers that we must keep in any cloud API or stack is that if they aren’t supported, they do not negatively impact the ability to invoke the service. They are ignored by applications which do not support them. Only through commoditization and a common model can this come to fruition.
Having an API is important. It’s what makes integration of applications, infrastructure, and ultimately clouds possible. But it isn’t the definition of that interface across disparate implementations of similar technology that will make or break intercloud. What will make or break intercloud is the definition of a consistent semantic model for core services and components that can be used to describe the technologies and policies and meta-data necessary to enable interoperability.
The news came recently that Novell will be developing a cloud-computing platform with Tencent, China's largest Internet service provider. As part of the deal, the two companies will establish a research laboratory in Shenzen, China that will be used to develop the data center network that will serve as the cloud platform.
These kinds of deals often seem distant as our knowledge of China's Internet community is a bit limited. But sometimes news comes out that provides a whole new insight.
That's exactly what happened this week when China Computerworld dropped the F-bomb on Tencent in an article and featured the company's mascot bleeding from knife wounds.
Sponsor
Whoa! And we thought the media here can be tough.
The article stems from Tencent's controversial place in the market. It has been accused of imitating its rivals then bullying them out of the market.
Danwei.org, posted an English version of the story. Here's an excerpt:
"Tencent is never the first to eat crab' [to try out new things]. It looks for a space in a mature markets to shove its way in. However, the methods it chooses also invite controversy: imitation, sometimes unscrupulous 'shanzhai' copying.As early as 2006, Sina founder Wang Zhidong openly accused [Tencent founder] Ma Huateng of being the industry's 'plagiarism king,' and of brazen plagiarism at that. Similar voices have been heard in the years since. Most recently, Data Center of the China Internet (DCCI) director Hu Yanping questioned Tencent's creative abilities, saying that it was not an outstanding innovator, and was actually the mortal enemy of innovation among smaller Internet enterprises."
And of course, Tencent had its own rebuttal:
"However, the China Computerworld feature story, without conducting any interviews with Tencent, used crude language against a responsible enterprise and used a disgusting illustration to damage our trademark and corporate image, creating an extremely adverse reaction and rudely hurting the feelings of the vast numbers of ordinary Tencent users. We strongly condemn this action and reserve the right to take legal action to protect our rights."
The article does not mention Tencent's new cloud initiative. But it still is an interesting perspective about a company that rivals in size any hosting provider around the globe.
It also reflects on the challenges that a foreign company faces when partnering with a major Chinese corporation that without a doubt has its own enemies watching its every move.
Discuss
Two reports issued today by Gartner and IDC show a SaaS market exhibiting double digit growth across the enterprise and in the overall market.
This a SaaS market dominated by vendors that sell services more than on-premise licenses. The growth is proof that the IT kings of the enterprise face the greatest potential disruption as traditional licensing models are replaced by subscription services.
Sponsor
Gartner is forecasting the enterprise application software market to surpass $8.5 billion in 2010. That is up 14.1% compared to 2009 when revenues hit $7.5 billion.
IDC is reporting that the SaaS market had worldwide revenues of $13.1 billion in 2009. The research firm estimates these revenues will reach $40.5 billion by 2014.
Enterprise SaaS services represent 10% of the overall enterprise software market. The SaaS market's share will rise to 16% by 2014.
Gartner reported that the SaaS market had some attrition in 2009 but revenues were generally up. Continued growth will be fueled by the attention on cloud computing.
What we see are markets with a lot of activity and others that are still uncertain of SaaS and its uses in the cloud. Gartner points out that security is not as much of an issue as it used to be, and that's reflected in the increasing use of collaboration technologies. But there are places in the enterprise where the cloud is a concern more for its data portability issues than anything else. In resource planning, for instance, companies have deeper concerns about using a SaaS provider.
ICD's report reflects how quickly the market has moved from software delivered on a compact disc.
For instance, IDC estimates that by 2012, about 85% of new software to the market will be delivered as a service. Revenues from SaaS services will account for nearly 26% of net new growth in the software market in 2014. In 2010, IDC predicts a $7 billion drop in worldwide license revenues.
What this all means is that there will be a significant shift for traditional enterprise vendors. The last 20 years have seen the rise of the IT kings like Oracle, Microsoft and a host of others. They now face a challenge to their crowns.
And that is just in one year.
Discuss
My previous trips to China, India, Japan, London have been super productive. I get a chance to meet tons of new people, make a lot of friends and talk about something that I am truly passionate about: Cloud Architectures and Amazon Web Services Cloud.
Next month, I will be in Brazil and traveling to 3 main cities to keynote and present at different conferences and user groups. My complete plan is as follows:
Aug 6 - Aug 11, 2010 in Sao Paulo:
Aug 12 - Aug 16, 2010 in Rio De Janero:
Aug 17 - Aug 21, 2010 in Brasilia:
If you are in Brazil and passionate about cloud computing, I would like to meet you. If you are an aspiring cloud developer or architect, system integrator trying to win
a local SaaS contract or an ISV trying to build a cloud strategy around your
product, send me an email at evangelists [[at]] amazon [[dot]] com to schedule a meeting. I
would love to exchange ideas, learn more about the local market and
discuss the future. If you are a leader of a local user group and would like us to present to your group, please contact me in advance.
-- Jinesh
The LA Times reported over the weekend that Google had missed its deadline for implementing a the city's new email system, in part because it had not fully responded to some of the LA Police Department's security concerns.
But Google announced this morning that it was introducing a new edition of Google Apps, Google Apps for Government, an indication that Google is still committed to making inroads into enterprise, schools, and now government.
Sponsor
Google Apps boasts Federal Information Security Management Act (FISMA) certification from the U.S. Government - the first suite of cloud computing applications to receive it. The FISMA law applies to all systems utilized by federal government agencies, and Google's accreditation means that the federal government has reviewed Google's security controls. "This review," says Google in the blog post announcing the Apps for Government edition, "makes it easier for federal agencies to compare our security features to those of their existing systems; most agencies we have worked with have found that Google Apps provides at least equivalent, if not better, security than they have today. This means government customers can move to the cloud with confidence."
Google Apps for Government stores Gmail and Calendar data in a separate storage system housed state-side, one designed exclusively for Google's government contractors. Google Apps for Government is available to any federal, state or local government in the U.S., and according to Google "should give governments an even stronger case for making the move to the cloud."
Whether or not it's a strong enough case for the Los Angeles Police Department, and other skeptics, remains to be seen.
Discuss
 | A compendium of Windows Azure, Windows Azure Platform Appliance, SQL Azure Database, AppFabric and other cloud-computing articles. |
Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:
To use the above links, first click the post’s title to display the single article you want to navigate.
Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)
Read the detailed TOC here (PDF) and download the sample code here.
Discuss the book on its WROX P2P Forum.
See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.
Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.
You can now freely download by FTP and save the following two online-only PDF chapters of Cloud Computing with the Windows Azure Platform, which have been updated for SQL Azure’s January 4, 2010 commercial release:
HTTP downloads of the two chapters are available for download at no charge from the book's Code Download page.
Jai Haridas of the Windows Azure Storage Team explains How WCF Data Service Changes in OS 1.4 Affects Windows Azure Table Clients in this 7/26/2010 post:
The release of Guest OS 1.4 contains an update to .NET 3.5 SP1 which contains some bug fixes to WCF Data Services. We have received some feedback on backward compatibility issues in Windows Azure Tables with respect to the WCF Data Services update. The purpose of this post is to go over some of the breaking changes when moving from .NET 3.5/.NET 3.5 SP1 to the above mentioned update to .NET 3.5 SP1. In addition, we hope this also helps when you upgrade your application to use .NET 4.0 from .NET 3.5 SP1, since the same breaking changes are present in .NET 4.0 too.
Issue #1 - PartitionKey/RowKey ordering in Single Entity query
Before .NET 4.0 and the update to WCF Data Services in .NET 3.5SP1, no exception was thrown when trying to get a single entity which does not exist in the storage service. Take for example the following LINQ query, where the RowKey match is in the expression before the PartitionKey match:
var q = from entity in context.CreateQuery<MyEntity>(“MyTable”) where entity.RowKey == “Bar” && entity.PartitionKey == “Foo” select entityThis would generate the following Uri:
http://myacocunt.table.core.windows.net/MyTable?$filter=PartitionKey eq ‘Foo’ and RowKey eq ‘Bar’If the entity did not exist, the use of $filter to specify the filter did not result in an exception and an empty result set was returned before the update. The bug is that the above query should have resulted in the following Uri:
http://myacocunt.table.core.windows.net/MyTable(PartitionKey=“Foo”,RowKey=”Bar”)This Uri format always results in a DataServiceQueryException with error code “ResourceNotFound” when the entity does not exist.
In the update, when querying for a single entity if the RowKey is filtered before the PartitionKey in the query, it now results in the above Uri which addresses a single entity (i.e. $filter is not used). An exception is now raised if the entity is not present on the server irrespective of the order in which the keys are specified in the LINQ query.
Note, the following LINQ query:
var q = from entity in context.CreateQuery<MyEntity>(“MyTable”) where entity.PartitionKey == “Foo” && entity.RowKey == “Bar” select entityhas always resulted in the following query Uri:
http://myacocunt.table.core.windows.net/MyTable(PartitionKey=“Foo”,RowKey=”Bar”)which always results in a DataServiceQueryException with error code “ResourceNotFound” when the entity does not exist. This has always been the case and has not changed.
Any dependency on the behavior that an empty set will be returned when an entity is not found will break your application because the new behavior is to raise an exception even when RowKey precedes PartitionKey in the LINQ query.
The suggestion from WCF Data Services team for this breaking change is:
- In update to .NET 3.5 SP1 (available in the Guest OS 1.4 release) and in .NET 4.0, a new flag “IgnoreResourceNotFoundException” on the context is provided to control this. Use IgnoreResourceNotFoundException to ignore exceptions by specifying the following:
context.IgnoreResourceNotFoundException = true;- Always Catch exceptions and then ignore ”Resource Not Found” exceptions if required by your application logic.
Issue #2 - Uri double escaping that impacts queries
Previous versions of the WCF Data Services library did not escape certain characters when forming the Uri. This allowed some entities to be inserted but not retrieved or deleted. A blog post covered the characters that had problems. The solution was to encode/escape them before using them. However, the updated .NET 3.5SP1 (used in OS 1.4) and .NET 4.0 has fixed this issue by using the appropriate encoded/escaped value. This now would break existing applications that had already escaped their values.
The resolution is to review your application to see if values are being escaped and undo these changes to see if it works with the WCF Data Service release.
Issue #3 - Uri escaping that impacts AttachTo
The DataServiceContext tracks entities using its address when entities are either returned in the query result or when AddObject/AttachTo is invoked. The address is basically the Uri that contains PartitionKey and RowKey. For example, an entity with PartitionKey=foo and RowKey=bar is tracked using
http://myaccount.table.core.windows.net/MyTable(PartitionKey='foo',RowKey='bar').With the Uri escaping fix mentioned above, the previous version (pre .NET 3.5SP1 update and pre .NET 4.0) has a mismatch in how it creates this address to track the entities when the address has a special character in it, which needs to be escaped. The mismatch is between the address (that is appropriately escaped) it uses for an entity it receives from the server and the address it uses (that is un-escaped) when AddObject/AttachObject is invoked. This mismatch causes entities with same key to be tracked twice.
For example, for an entity with PartitionKey = ‘foo@bar.com’ and RowKey = ‘’, the address used to search the list of tracked entities at the time of AttachTo and AddObject is:
.NET 3.5 SP1 update and .NET 4.0 uses the same address that is used to track the entity:
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai%40com',RowKey='')Pre .NET 4.0 and .NET 3.5 SP1 update however uses a different (un-escaped) address:
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai@com',RowKey='')So let us use an example to see exactly where the inconsistency is:
New Client Library (update to .NET 3.5 SP1 and .NET 4.0):
When entity returned from server as result of a query, server returns ID that is escaped:
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai%40com',RowKey='')
and WCF Data Service Client library tracks using this id for the entity.Then assume an AddObject/AttachTo is invoked for an object with the same key, so the WCF Data Service Client library uses the escaped URI to try to add/attach the object:
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai%40com',RowKey='')This resuls in an InvalidOperationException exception being thrown with the message “Context is already tracking a different entity with the same resource Uri”. This is the behavior that the client library wants, since the object was already being tracked in the context, so the program should not be able add/attach another object with the same key.
Old Client Library:
Now let us look at the example using the client library before the update. When entity returned from server as result of a query, server returns the ID that is escaped:
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai%40com',RowKey='')
and WCF Data Service Client library tracks using this id for the entity.Then when the AddObject/AttachTo is invoked, the WCF Data Service Client library does not escape it and uses
http://myaccount.table.core.windows.net/Emails(PartitionKey='jai@com',RowKey='')
to track the newly added object and hence causing the inconsistency. Instead, the client library should have escaped the keys in order to know that it was already tracking an object of that name, which is what the update in the new client library now does.For the old client library, this can lead to strange behavior since two instances that represent the single server entity will be tracked in a single context (one is tracked via a query result and the other is tracked via either AttachTo or AddObject)
- If both instances are unconditionally updated, the user may inadvertently lose some changes.
- Let us assume a scenario where a table is used like a lookup table. An application may choose to query all entities from this lookup table with the context tracking these entities. Here the context uses IDs that are appropriately escaped. Now an application may rely on “Context is already tracking…” exception when it adds a new entity. However, the bug can cause the context to track it using an un-escaped URI and the collision is not detected during AddObject and the context tracks two instances that represent the same key. When SaveChanges is invoked, the server fails because the entity already exists on the server and the server correctly returns “Conflict”. However, an application may not be expecting this behavior since it expected the conflict to be detected while “AddObject” was invoked rather than SaveChanges.
- If conditional update is used on both instances, only first update that is processed by the server will succeed and second will fail because of ETag check. The order in which the entities are added to the context (via query or AddObject/AttachTo) will determine the order of requests dispatched to the server.
However, if the address does not contain special characters, then AddObject and AttachTo would throw InvalidOperationException with message “Context is already tracking a different entity with the same resource Uri”, and everything would work fine in the old client library.
This bug has been fixed in the .NET 3.5 SP1 update and in .NET 4.0 where the context escapes the Uri even when AddObject/AttachTo is invoked hence recreating the same address (and hence correctly leading to an InvalidOperationException mentioned above).
Let us go over this using a code example to show how the issue could occur:
TableServiceContext context = tableClient.GetDataServiceContext(); // For simplicity we have ignored the code that uses CloudTableQuery to // handle continuation tokens. var q = from entity in context.CreateQuery<MyEntity>("Emails") select entity; // Let us assume entityInTable is an already existing entity in table retrieved // using the above query and will now be tracked by the context. var entityInTable = q.FirstOrDefault(); // now let us create a new instance but with the same PartitionKey and RowKey var someEntity = new MyEntity { PartitionKey = entityInTable.PartitionKey, RowKey = entityInTable.RowKey }; try { // NOTE: Depending on WCF release and key values, AttachTo may throw // an InvalidOperationException with message: // "The context is already tracking a different entity with the same resource Uri." // CASE 1> Pre .NET 3.5SP1 update => Depending on the key value, an exception may be thrown. // If the key contains a character that needs to be encoded, // then an exception is NOT thrown, otherwise, an exception is // always thrown. // CASE 2> .NET 3.5SP1 update and .NET 4.0 => An exception is always thrown // // Example of an value: If PartitionKey = ‘foo@bar.com’ and RowKey = ‘’ then // the entity is tracked as: // http://myaccount.table.core.windows.net/MyTable(PartitionKey='foo%40bar.com',RowKey='') // However, when attaching a new object in CASE 1, the id is not escaped and hence // the duplicate entity is not tracked and an exception is not thrown. // leading to strange behavior if the application unconditionally updates both the instances context.AttachTo("Emails", someEntity, "*"); } catch (InvalidOperationException e) { // Check if message is "The context is already tracking a different entity with the // same resource Uri." and handle this case as required by your application } context.UpdateObject(someEntity); context.SaveChanges();The resolution is to upgrade the WCF Data Services library. However, after upgrading, you should ensure that your code handles exceptions. This is one of the recommended best practices.
One can also check for entity existence using key equality check rather than instance equality before attaching/adding a new object instance. So in the example below, the first LINQ query finds the tracked entity but the second one does not since it does an equality check on reference which it is not the same. If we Attach only if an entity is not found using the first LINQ query, we will never have duplicates. Also, remember that WCD Data Service recommends that a new instance of context be used for every logical operation. Using a new context for every logical operation should reduce the chances of tracking duplicate entities.
Example:
// Create a new instance and let entityInTable represent an entity retrieved via a query var someEntity = new MyEntity { PartitionKey = entityInTable.PartitionKey, RowKey = entityInTable.RowKey }; // This will find the tracked entity instance since we are looking for key equality. If // trackedEntityKeySearch is not null, it means entity is tracked so do not invoke AddObject/AttachTo var trackedEntityKeySearch = (from e in context.Entities where ((TableServiceEntity)e.Entity).PartitionKey == someEntity.PartitionKey && ((TableServiceEntity)e.Entity).RowKey == someEntity.RowKey select ((TableServiceEntity)e.Entity)).FirstOrDefault<TableServiceEntity>(); // NOTE: This will not find the tracked entity even if it is tracked since it is not the same // object instance. So the above query is preferred to see if a particular entity is being tracked var trackedEntityReferenceSearch = (from e in context.Entities where e.Entity == someEntity select ((TableServiceEntity)e.Entity)).FirstOrDefault<TableServiceEntity>();We apologize for any inconvenience this has caused and hope this helps you make a smooth transition as possible to .NET 3.5 SP1 update or .NET 4.0. However, we would like to end this by reiterating a couple of best practices that the WCF Data Service team recommends:
- Always handle exceptions in AddObject, UpdateObject, AttachTo and in Queries
- DataServiceContext is not thread safe. It is recommended to create a new context for every logical operation
We will have more on best practices in the near future.
Jai Haridas
Wayne Walter Berry’s Getting Started With Project Houston: Part 1 to the SQL Azure Team blog of 7/26/2010 covers the startup basics:
Microsoft Project Code-Named “Houston” (Houston) is a light weight database management tool for SQL Azure. Houston can be used for basic database management tasks like authoring and executing queries, designing and editing a database schema, and editing table data. Currently, Houston is its first community technology preview (CTP), the instructions and screen shots shown in this blog post are from the CTP version of Houston. In this blog post I am going to show how to get started using Houston.
Houston is a web based, Silverlight application, when means that you can access and use it from any web browser that supports Silverlight (for a list see this web page), anywhere you have an Internet connection. Houston isn’t software that you install, however Silverlight is. You first need to have Silverlight installed to use Houston. If you don’t have it installed, you will be prompted to install it when you access Houston for the first time.
You can start using Houston by going to: https://manage.sqlazurelabs.com/. SQL Azure labs is the location for projects that are either in CTP or incubation form. The URL will probably change on release.
Logging In
Currently for the CTP you need to enter your SQL Azure server, database, administrative login and password to when logging in.
Remember, Houston is a Silverlight application, it is running locally client side on your computer. It is not a web application, but it does communicate with web services hosted within Windows Azure.
The Start Page
Once you are logged in you are presented front and center with the start page. The start page consists of a rotating cube that displays information about the database you choose to access. One thing to note is that Houston only allows you to manage one database in the browser windows. If you want to manage more than one database, use the tabs on your browser to open more than one instance of Houston.
Click on the arrows to rotate through the spinning box, make sure to say “oh” and “ah” as it rotates.
On the help page, note the links to the Houston videos that will give you further help.
Navigation
At the top of the application is a toolbar that changes depending what is being displayed in the main tab pane.
The database toolbar appears like this:
The table toolbar changes to look like this:
You can jump back to the database toolbar at any time by clicking on the Database link at the top left of the application, regardless of whether you are done working within the current tab.
One thing to note is that Houston maintains the state of your current changes as you navigate between different tabs. If you modify a table, then open another one without saving your change, you change is not lost. The application indicates that the table/tab needs to be saved by writing a pencil icon inside the tab.
If you try to exit the application you are warned that there are changes that need to be saved, so that you don’t lose your work
The change tracking that Houston performs is a nice touch that allows you to work on several tables and their corresponding stored procedures at the same time. Because it feels like you are using a HTML application, it might not feel like change tracking would be in place, however the functionality is very similar to SQL Server Management Studio.
Just a reminder, Houston is in CTP. Because of this, there will be various bugs in the product still, like the spelling error in the dialog above. I reported it already via Microsoft Connect and we would appreciate your bug reports and feedback on this product.
Feedback or Bugs?
This release of project Houston is not supported by standard Microsoft support services. For community-based support, post a question to the SQL Azure Labs MSDN forums. The product team will do its best to answer any questions posted there.
To provide feedback or log a bug about project Houston in this release, use the following steps:
- Navigate to Https://connect.microsoft.com/SQLServer/Feedback
- You will be prompted to search our existing feedback to verify your issue has not already been submitted.
- Once you verify that your issue has not been submitted, scroll down the page and click on the orange Submit Feedback button in the left-hand navigation bar.
- On the bug form, select Version = Houston build CTP 1 - 10.50.9610.34.
- On the bug form, select Category = Tools (SSMS, Agent, Profiler, Migration, etc.).
- Complete your request.
If you have any questions about the feedback submission process or about accessing the portal, send us an email message: sqlconne@microsoft.com.
Summary
This is just the beginning of our Microsoft Project Code-Named “Houston” (Houston) blog posts, make sure to subscribe to the RSS feed to be alerted as we post more information.
For examples of working with table, query, stored procedure and view objects, see my Test Drive Project “Houston” CTP1 with SQL Azure post of 7/23/2010.
Kellandved explains Deploying phpBB on Windows [and SQL] Azure in this 7/26/2010 post to the phpBB blog:
Windows Azure is the Mircrosoft cloud computing solution. One thing that might come as a surprise is that is specifically intended to run php applications (well, it runs .net too, even java). Long story short, we got introduced to the azure platform during the JumpIn! Camp. The project to get phpBB running on the Azure platform started right there – now it’s showing results.
The pre[re]quisite for deploying on Azure is the new support for the native MSSQL driver, which was contributed by Microsoft.
However, a few issues had to be tackled to actually run phpBB in the cloud, namely:
SQL Azure is not quite the same as SQL Server and requires a few tweaks
- Files (uploads) can’t be stored on the local file system, but have to be shared among all instances
- A few oddities of the Azure platform, especially regarding values reported by the webserver
To expand a bit on the issues: SQL Azure requires primary (“clustered”) keys on all tables, something the default phpBB schema does not deliver. On the flip side, the phpBB MSSQL schema includes partition clauses, which are not supported on Azure. Long story short: it needs a different schema. Files cannot be stored locally in a multi-server environment – for instance a cloud – but have to be shared between all instances. This required a few patches to the phpBB core. Finally, Azure includes a load balancer, which reports an incorrect – internal – port via the ‘SERVER_PORT’ variable. This had to be corrected.
All of these changes can be found in my azure branches of my fork at github: http://github.com/kellanved/phpbb3/tree/feature/azure_blob_storage and http://github.com/kellanved/phpbb3/tree/bug/9725
So, how to use these? The answer is:
- Download the source from the bug/9725 branch
- Create a SQL Azure database
- Add a firewall rule to allow your own machine to connect to the SQL Azure database
- Start the phpBB installation, using the bug/9725 branch on your local machine
- Use the credentials of the SQL Azure database during the install
- Add the line define(‘AZURE_INSTALL’, true); to the config.php file
- Delete the install directory
- Create an Azure package with the phpBB installation you had locally
- Deploy the package on Azure
- Add a firewall rule to SQL Azure to allow your phpBB instance to connect with the database
Congrats, you’re running phpBB on Azure.
Jamesy (@bondigeek) claims OData – Multi-dimensional javascript arrays made easy in this 7/26/2010 post:
This might seem like an obvious thing to some but it struck me just how bloody easy it is to create multi-dimensional arrays in javascript when you’re using OData.
Pretty much you don’t need to do anything except make your OData call and it will take care of the rest.
As is always the way I was working on a project yesterday that inspired this blog post and solving a problem where my Ajax calls were getting a tad too chatty and slowing down the performance of my context menus.
Read on for more…
Setting the scene
In my project I have a menu list of files and folders and each time you select one of the files the display is updated and the context menus re-configured depending on the state of the selected item.
This in turn was triggering a change on a dropdown menu that populated an unordered list of groups and categories.
Initially I was making the call to get a list of categories and groups. Each group itself has child records of type ImageOutput and each category has child records of type ImageOutput.
When a group was selected it would pop across to the server and get the list of ImageOutputs. This could happen many, many times but the Groups, Categories and ImageOutput lists will rarely, if ever change.
So caching this on the client side in a multidimensional array was an obvious choice.
OData to the rescue
So to solve this problem I could have manually created the array and made all the appropriate calls to the server to get each set of lists but that would involve multiple round trips to the server. Why not do it the easy way?
First off I have my helper function to make the Ajax calls:
function GetListData(url,async){ var list = ""; $.ajax({ type: "GET", url: url, async: async, contentType: "application/json; charset=utf-8", dataType: "json", success: function (msg) { list = msg.d; }, error: function (xhr) { ShowError(xhr); } }); return list; }The above code just wraps the call to my OData urls and returns the results as json.
Armed with the wrapper function I just need make my calls to the OData urls that expose the data I am after. The first call to ImageOutputCategories just gets a simple list of categories.
The second call uses the $expand system query option which tells it to not only get the OutputGroups but also to return the associated ImageOutputs for each OutputGroup.
You can read more about the $expand system query option here.
if (categories == null) { categories = GetListData("/DataServices/AMPLibrary.svc/ImageOutputCategories", false); } if (groups == null) { groups = GetListData("/DataServices/AMPLibrary.svc/ImageOutputGroups?$expand=ImageOutputs", false); }This second call will return me a multi-dimensional array that I can store on the client in memory and access whenever I please, no trip to the server required.
Some sample code below illustrates how the multi-dimensional array is used:
Populate the Group List:
$("#output-group").children().remove(); for (var i = 0; i <= groups.length - 1; i++) { var option = "<option value='" + groups[i]["Id"] + "'>" + groups[i]["ImageOutputGroupName"] + "</option>"; $("#output-group").append(option); if (groups[i]["Id"] == $("#output-group").val()) { cropHeight = groups[i].CropHeight; cropWidth = groups[i].CropWidth; GetImageGroupOutputs(groups[i]["ImageOutputs"]); } }Populate the OutputGroup list for the selected Group:
function GetImageGroupOutputs(imageOutputs){ $(".image-formats-row").children().remove(); var checkAll = '<div><input id="check-all-formats" type="checkbox" /><label class="check-all" id="all-formats-label">Check all</label></div>'; $(".image-formats-row").append(checkAll); for (var i = 0; i <= categories.length - 1; i++) { if (imageOutputs.length>0) { var ul = "<ul>" for (var k = 0; k <= imageOutputs.length - 1; k++) { if (imageOutputs[k]["ImageOutputCategoryId"] == categories[i]["Id"]) { var title = "<span class='category-title'>" + categories[i]["CategoryName"] + "</span>"; $(".image-formats-row").append(title); break; } } for (var k = 0; k <= imageOutputs.length - 1; k++) { if (imageOutputs[k]["ImageOutputCategoryId"] == categories[i]["Id"]) { var li = "<li><input id='OutputFormat_" + imageOutputs[k]["Id"] + "' type='checkbox' outputid='" + imageOutputs[k]["Id"] + "'>"; li += "<label class='output-format-name'>" + imageOutputs[k]["OutputName"] + "</label></li>"; ul += li; } } ul+="</ul>"; $(".image-formats-row").append(ul); } } }LOVE IT, LOVE IT, LOVE IT.
Thanks OData Team.
BondiGeek
Kevin Kell describes Migrating a SQL Server Database to SQL Azure in his 7/25/2010 post to Learning Tree’s Cloud Computing blog:
As a follow on to my colleague’s recent excellent post I thought that this week I would present a practical, hands-on example of moving a real-world on-premise SQL Server database to SQL Azure.
There are at least three ways to migrate data into SQL Azure:
SQL Script
- Bulkcopy (bcp)
- SQL Server Integration Services (SSIS)
Each has benefits and limitations. Don’t use the script option to move very large data volumes, for example.
Here we are going to take an approach based on kind of a combination of 1 and 3. We will script our database schema and then use SSIS to replicate the data to the cloud. Since there are some features in SQL Server 2008 that are not supported in SQL Azure we will have to do a little hand work to modify the code that is generated for us. We can minimize that hand work if we make some changes in the default options the wizard gives us before we generate the script.
Here is what I had to do:
In SQL Server Management Studio, right click the on-premise database
Choose Tasks | Generate Scripts
Change the default options so that
- ANSI Padding = False
- Convert UDDT to Base Types = True
- Set Extended Properties = False
- USE DATABASE = False
Delete all the stuff having to do with creating the database, etc. I have already created the database in the SQL Azure Developer Portal.
Delete the unsupported features. In my case these included:
- SET ANSI_NULLS ON
- SET ANSI_NULLS OFF
- ON [PRIMARY]
- NOT FOR REPLICATION
- PAD_INDEX = OFF, (
- WITH ( …
- TEXTIMAGE
- NONCLUSTERED
I moved some things around in the code so that tables were created before views, etc. Stuff like that just seemed to make sense to me. You also have to move the creation of things that other things depend on higher up in the code.
- I went through an iterative process of running the code, examining the errors, making changes, and running the code again. Yeah, it is a lot of code. It took me about 30 minutes to get it right. Your time may vary.
Once the schema exists on SQL Azure it is straightforward replicate the data with SSIS. The wizard pretty much takes care of everything!
Click here to view the screencast:
http://www.youtube.com/watch?v=SW3TcF4W1Ws
Okay, so perhaps it is a little more tedious than one would like. Somebody has to actually look at the code. That, I think, is part of the developer’s job. Yes, maybe it should be easier and maybe someday it will be but for now it is what it is. There is an interesting project on CodePlex that attempts to further automate this process. I recommend that you check it out.
If you get a chance, try this out yourself! Then consider attending Learning Tree’s Windows Azure Course!
I don’t know why Kevin would select the SQL Server Migration Wizard for the last choice. SQLMW uses BCP for bulk inserts and saves more than half the time of any other technique.
<Return to section navigation list>
Radiant Logic offers a downloadable Gartner Report: "The Emerging Architecture of Identity Management" white paper with this description:
Your current identity management infrastructure is built for a world that’s changing quickly—one based on pushing identity from the center, instead of pulling it from many disparate sources. But today’s centralized identity infrastructure can’t keep up with tomorrow’s increasingly federated demands, from user-centric identity, Identity-as-a-service, and the cloud. Luckily, Gartner has outlined a vision for tomorrow’s identity landscape.
Vice President and Research Director Bob Blakley’s groundbreaking new paper offers a roadmap for a new Identity Management infrastructure—one with virtualization at its core. Don’t miss this free resource!
<Return to section navigation list>
Richard Seroter makes a Cloud Provider Request: Notification of Exceeded Cost Threshold in this 7/26/2010 post:
Ideally, I’d love a way to set up a complex condition based on various sub-services or types of charges. For instance, If bandwidth exceeds X, or Azure AppFabric exceeds Y, then send me an SMS message. But I’m easy, I’d be thrilled if Microsoft emailed me the minute I spent more than $20 on anything related to Azure. Can this be that hard? I would think that cloud providers are constantly accruing my usage (bandwidth, compute cycles, storage) and could use an event driven architecture to send off events for computation at regular intervals.
If I’m being greedy, I want this for ANY variable-usage bill in my life. If you got an email during the summer from your electric company that said “Hey Frosty, you might want to turn off the air conditioner since it’s ten days into the billing cycle and you’ve already rung up a bill equal to last month’s total”, wouldn’t you alter your behavior? Why are most providers stuck in a classic BI model (find out things whenever reports are run) vs. a more event-driven model? Surprise bills should be a thing of the past.
Are you familiar with any providers who let you set charge limits or proactively send notifications? Let’s make this happen, please.
Rob Blackwell’s AzureRunMe project Release 1 of 7/24/2010 on CodePlex lets you “[r]un your Java, Ruby, Python, Clojure or (insert language of your choice) project on Windows Azure Compute:”
Latest
Also now runs Tomcat - contact me if you want to know how, but it's much neater than the TomCat Solution Accelerator because your WAR files can just come from Blob Store!
I've used it to run Restlet and also Jetty (although without the NIO support).Introduction
There are a number of code samples that show how to run Java, Ruby, Python, etc. on Windows Azure, but they all vary in approach and complexity. I thought there ought to be a simplified, standardised way.
I wanted something simple that took a self contained ZIP file, unpacked it and just executed a batch file, passing the HTTP port as an argument.
I wanted ZIP files to be stored in Blob store to allow them to be easily updated with all Configuration settings in the Azure Service Configuration.
Trace messages, debug, console out and exceptions are easiest to surface via the ServiceBus.
Prerequisites
(see http://msdn.microsoft.com/en-us/windowsazure/cc974146.aspx )
- The Windows Azure SDK & Tools for Visual Studio
- The Windows Azure AppFabric
Instructions
Organise your project so that it can all run from under one directory and has a batch file at the top level.
In my case, I have a directory called c:\foo. Under that I have copied the Java Runtime JRE. I have my JAR files in a subdirectory called test and a runme.bat above those that looks like this:
cd test
..\jre\bin\java -cp Test.jar;lib\* Test %1I can bring up a console window using cmd and change directory in
c:\fooThen I can try things out locally by typing
C:>Foo> runme.bat 8080The application runs and serves a web page on port 8080.
I package the jre directory as jre.zip and the test directory along with the runme.bat file together as dist.zip.
Having two ZIP files saves me time - I don't have to keep uploading the JRE each time I change my Java application.
My colleague has a ruby.zip file containing Ruby and Mongrel and his web application in rubyapp.zip in a similar way.
Upload the zip files to blob store. Create a container called "packages" and put them in there. The easiest way to do this is via Cerebrata Cloud Studio http://clumsyleaf.com/products/cloudxplorer.
Another alternative is to use the UploadBlob command line app distributed with this project.
The next step is to build and deploy Azure RunMe ..
- Load Azure RunMe in Visual Studio and build.
- Change the ServiceConfiguration.cscfg file.
- Update DataConnectionString with your Windows Azure Storage account details so that AzureRunme can get ZIP files from Blob store.
- Change the TraceConnectionString to your appFabric Service Bus credentials so that you can use the CloudTraceListener to trace your applications.
- By default, Packages is set to "packages\jre.zip;packages\dist.zip" which means download and extract jre.zip then download and extract dist.zip, before executing runme.bat
- Click on AzureRunMe and Publish your Azure package.
- Sign into the Windows Azure Developer Portal at http://windows.azure.com
- Create a New Hosted Service and upload the package and config to your Windows Azure account. You are nearly ready to go.
- Change the app.config file for TraceConsole to include your own service bus credentials.
- Run the TraceConsole locally on your desktop machine.
- Now run the Azure instance by clicking on Run in the Windows Azure Developer Portal.
Deployment might take some time (maybe 10 minutes or more), but after a while you should see trace information start spewing out in your console app. You should see that it's downloading your ZIP files and extracting the[m]. Finally it should run your runme.bat file.
If all goes well your app should now be running in the cloud!
Future Ideas
- Look at changing the TraceListener to use Hybrid mode and reducing service bus costs
- Consider attaching to an X Drive for persistent storage (via config)
- Try to find a way of exposing an internal end point via the ServiceBus to allow Clojure users to hook up a SLIME / SWANK connection.
Credits
- This project uses Ionic Zip library, part of a CodePlex project at http://www.codeplex.com/DotNetZip which is distributed under the terms of the Microsoft Public License.
- TraceConsole and TraceListener are code samples from the Microosft appFabric SDK (with minor modifications).
Rob Blackwell
July 2010
Joel Jeffery’s Microsoft Azure Cloud Services and Mobile Applications post of 7/22/2010 reviews Jason Zander’s presentation at UK Tech days:
I’ve been following the Microsoft cloud service offerings for a couple of years since the Microsoft Architect Insight Conference 2008. What started as SQL Server Data Services and the nascent BizTalk Services, has now grown into an offering that in my opinion does not just complete with Amazon and Google cloud services, but far exceeds the capabilities of their model.
Here’s a screenshot of a really simple ASP.NET page that I’m going to run in the local Azure test harness. You need to make sure it’s running as Administrator, or you can’t launch the Azure Simulation Environment.
If I’ve done all that, I can hit F5 and get this:
Shortly followed by the web application as if it were running in the cloud:
In the http://www.youtube.com/watch?v=ava6yFMewN8'>video, Jason builds a quick application connecting some .NET entity classes to a SQL Azure instance and shows how quickly a simple web service can be deployed to the cloud.
You also get to see the developer tools for Visual Studio 2010 and Azure, including local simulations of the App Fabric and Dev Fabric infrastructure of Azure. In other words, you can test your Azure apps by running them on your local machine without needing to actually deploy them.
Instead of deploying an ASP.NET Web Application like I did above, Jason shows you the newly released Visual Studio Express and he walks through the steps to build a Silverlight mobile application to consume the service he just deployed on Azure.
The key takeaway here is just how easy it is to build and consume cloud services and applications across a broad spectrum of platforms. Follow this link for more information about Microsoft Azure, or give us a call a JFDI Phoenix in the UK.
Joel is a Microsoft Certified SharePoint 2010 specialist and Microsoft Certified Trainer.
The Microsoft Case Studies Team posted Lockheed Martin Merges Cloud Agility with Premises Control to Meet Customer Needs:
Headquartered in Bethesda, Maryland, Lockheed Martin is a global security company that employs about 136,000 people worldwide and is principally engaged in the research, design, development, manufacture, integration, and sustainment of advanced technology systems, products, and services. The company wanted to help its customers obtain the benefits of cloud computing, while balancing security, privacy, and confidentiality concerns. The company used the Windows Azure platform to develop the Thundercloud™ design pattern, which integrates on-premises infrastructure with compute, storage, and application services in the cloud. Now, Lockheed Martin can provide its customers with vast computing power, enhanced business agility, and reduced costs of application infrastructure, while maintaining full control of their data and security processes.
Organization Profile
With 136,000 employees and 2009 sales of U.S.$45.2 billion, Lockheed Martin operates in four business areas: Aeronautics, Space Systems, Electronic Systems, and Information Systems and Global Solutions.
Business Situation
Lockheed Martin wanted to deliver the performance and flexibility of cloud computing to its customers, while enabling them to balance security, privacy and confidentiality concerns.
Solution
Lockheed Martin used the Windows Azure platform to develop the Thundercloud™ design pattern, which integrates on-premises infrastructures with compute, storage, and application services in the cloud.
Benefits
- Agility and speed
- Enhanced infrastructure at lower costs
- On-demand, usage-based model
- Ubiquitous access
Return to section navigation list>
Ed Sperling interviews Mark McDonald, group vice president and head of research at Gartner Executive Programs to determine “[w]hy some IT organizations are progressing while others seem to be stuck in neutral” in this 7/26/2010 post to Forbes.com’s One On One colum:
What makes one CIO more successful than another is changing. In the past, deep knowledge of technology was critical. Now, a deep understanding of the business and how to move it forward has become the key metric.
That change is reflected not just in how much money the CIO's organization receives, but also who the CIO reports to. To find out what's driving these changes, Forbes caught up with Mark McDonald, group vice president and head of research at Gartner Executive Programs.
Forbes: What's changing across the CIO landscape?
Mark McDonald: The differences between rich and poor IT organizations are real and getting bigger.
How do you define rich and poor?
Richness is a level of IT performance as well as budget and standing in the enterprise. It's not just who has the most money. It's whether you're a strategic player and effective at what you do. These CIOs do tend to get more money, but they have a higher propensity to spend it more strategically. The richer IT organizations have been more proactive about cutting waste out of IT and they also do things faster. The poor organizations are hedging their bets across the whole organization
Is size of the overall corporation a factor?
No, it's more an understanding of how to create value in the enterprise. The poor IT organizations believe they create value by properly managing IT resources. In other words, "I'm proving to you that I don't waste the company's money." The rich IT organizations view it as, "Look at what we can do and how promptly we can be responsive to the business."
Does cloud-based computing level the playing field here?
No, absolutely not. There are two reasons. One is that the cloud is a giant red herring for IT organizations that are not rich because all they're looking at is cost arbitrage. That only gets you into trouble. You think you're solving a problem but you're just getting another one. We've seen a lot of people aggressively move applications into the cloud. They're moving 50% to 60% of applications that are not mission-critical. The poor organizations see this as a way of solving their infrastructure problems. The rich organizations are making this move to create more room for more value to the enterprise.
Using your terminology of rich and poor, do the rich get richer by doing this?
The rich are definitely getting richer, as measured in terms of budget, standing in the enterprise, creating the company strategy, and CIOs not reporting to CFOs.
What percentage of CIOs are on the rich side?
About 22%.
How many are in the middle?
Almost 30%. And there is a solid 50% that are poor and getting poorer.
What happens to that 50%? Do they get replaced or do the companies not realize they have a problem in the first place?
The reality is they'll be relegated to administrative irrelevance. That's the unifying theme. If you define your IT organization as enabling the business, that's an indication you're headed in the poorer direction. When you have organizations that talk about how IT contributes to the business and makes it transformational or directional, that's an early indicator the attitude of the CIO is pointed toward the richer side.
Page: 1, 2, 3, Next >
Lori MacVittie (@lmacvittie) asserts “When strategies are formed it quickly becomes obvious that cloud computing is more about balance than anything else” as a preface to her The Battle of Economy of Scale versus Control and Flexibility post of 7/26/2010 to F5’s DevCentral blog:
At a time when you’d think cloud computing would be the primary “go to” strategy for managing scale and rapid growth, multiple well-known and demanding organizations are building their own data centers instead.
With all the hype around cloud being faster, cheaper, and more efficient these folks must be crazy, right?
Not at all. In fact, these moves illustrate the growing friction between the economy of scale offered by cloud computing and the control and flexibility that is part and parcel of owning one’s own data center.
In April Twitter announced plans to build a data center of its own. On Wednesday it provided additional details on the Twitter Engineering blog.
“Later this year, Twitter is moving our technical operations infrastructure into a new, custom-built data center in the Salt Lake City area,” wrote Twitter’s Jean-Paul Cozzatti, who said having dedicated data centers will provide more capacity to accommodate growth of 300,000 new users per day. “Keeping pace with these users and their Twitter activity presents some unique and complex engineering challenges. Importantly, having our own data center will give us the flexibility to more quickly make adjustments as our infrastructure needs change.”
-- Data Center Knowledge, “Twitter Picks Utah for New Data Center”
Twitter isn’t the only Web 2.0 savvy organization moving to their own data center. Facebook earlier this year announced it, too, was also investing in building out its own data center.
BUT CLOUD AUTO-SCALES and STUFF!
It’s not all about scalability. I know that sounds nearly heretical, but it’s not. And it’s not a new mantra, either. Scalability is certainly a factor in why one would choose cloud computing over a localized deployment, but also important are control and flexibility.
Another consideration is the ability to customize your data center infrastructure to provide more granular control of operations. “That control gives us a ton of flexibility, and we can build new things without having to wait for our partner,” said Heiliger [Jonathan Heiliger, Facebook’s VP of Technical Operations]
-- Data Center Knowledge, “Data Centers: For When The Cloud is Not Enough”
If I’ve said it once I’ve said it a thousand times: control is a huge factor in the decision making process and something that isn’t effectively offered by today’s public cloud computing offerings. Remember the Information week analytics Cloud computing survey in 2009?
Even though security remains concern number one, control and configurability are on the top of the list, as well. The issue of control has almost always gone hand in hand with cloud adoption inhibitors, but it always takes a back seat to the more glamorous and scary “security” issue. These are not minor stumbling blocks in many cases, and the inability to rapidly adapt an infrastructure to meet growth and scale and make architectural changes, if necessary, are paramount to success. If cloud computing cannot provide the agility necessary to meet these challenges then it is logical to assume that organizations will either (a) stay in the local data center or (b) move to a local data center from the cloud when it becomes obvious the environment is inhibiting forward momentum.
Current adoption patterns indicate that this is not an anomaly, but will instead likely become the norm for organizations. Applications that are initially deployed “in the cloud” will, upon becoming a critical business application or growing beyond the meager means of control and flexibility offered by the cloud, will migrate to the data center, where control and agility are provided by the simple fact that the organization can change at will any piece of the infrastructure – from its physical implementation to its logical organization – at will. This is evident in the percentage of organizations using cloud for “dev and test” but not for production. Clearly the economy of scale and rapidity of deployment makes the cloud a perfect environment for development and testing but not necessarily production.
ECONOMY of SCALE MAY be INHIBITING SCALE
The irony is that the economy of scale offered by cloud may well be biting cloud in the proverbial derrière as it becomes the inhibitor to effective scale by limiting or making extremely difficult the architectural changes necessary for an application to scale in a cloud environment.
At some point scalability can become not about the application but about its infrastructure and the way in which that infrastructure interacts. It can become about the network and its components and how applications end up interacting with and through that infrastructure. In a cloud computing environment it is rarely the case that a customer can impact that infrastructure and, when it can, it is then limited by other factors such as underlying virtualization technology and the physical server infrastructure on which the application is ultimately deployed. If the answer to a scalability obstacle is more bandwidth and higher throughput, you can’t really add another NIC to a server in the cloud. That’s not your call. But it is if you’re in the data center, and it is virtualization – not cloud - that ultimately provides the agility to make such a change and rapidly propagate that change across the application deployment.
It isn’t always about costs. Well, okay, it is about cost but in IT it’s about cost as it relates to performance, or flexibility, or other operational functionality required to successfully meet data center and business goals. When spending less on infrastructure results in higher operational costs, the organization really hasn’t saved money at all. Savvy CIO and CTOs understand that it’s not a battle, but a balancing act. It’s not about achieving the highest economy of scale, but the best economy of scale given the specific operational and business needs.
Control is the obvious reason that Microsoft launched the Windows Azure Platform Appliance (WAPA) at WPC 2010.
RDA Corp. offers the following Key Reasons to Migrate to Azure whitepaper:
The Azure cloud computing platform offers tremendous advantages to organizations of all sizes. Here’s a list of key reasons to consider migrating, while you evaluate Azure and its benefits.
Focus on building apps, rather than being a data center
For small companies, especially startups, cash flow management is critical. Deploying a web application typically involves a significant hardware investment, as one considers redundancy, reliable Internet connections, load balancers, networking equipment, and any other infrastructure-related costs. Azure eliminates the need for such capital expenses and provides everything needed to deploy applications immediately, including application hosting, scaling, SQL hosting, and massive-scale storage.
Remove hardware and software requisition delays
For those organizations considering on-premise hosting, take into account the turnaround time typically associated with new hardware purchases. Between the specification, approval, ordering, delivery, configuration, and deployment activities, this could take a month or more. If any of these involve server software, such as Windows Server 2008 R2 or SQL Server 2008, there are licensing requisitions as well.
With Azure, there are no hardware specifications or licenses to deal with. Simply select the size of your virtual machine (VM), along with the number of instances, and your application is deployed to the Azure fabric in as little as 15 minutes. SQL Azure, the Azure-hosted version of SQL Server, is deployed in under a minute, with sizes up to 50GB.
Scale deployed applications without infrastructure expertise
Today’s applications have heavy demands placed on them during peak times. Sometimes traffic spikes are caused by advertisements, viral marketing, or other unexplained (and unexpected) reasons. Azure offers the ability to scale an application by simply specifying the number of VM instances an application is to be deployed to. The Azure platform takes care of everything else: load-balancing, multiple fault domains, health monitoring… Within minutes, your application’s capacity is increased to meet demand.
And when your traffic falls off, simply reduce the instance count, and Azure removes unneeded resources just as quickly.
Increase speed to market
Imagine having your Next Great Application idea. Now think of the lead-time for getting that idea up and running on a test platform. Then imagine the additional time to shift to a production environment. This is a typical challenge with on-premise or legacy hosting solutions. While the application is ready, there are servers, operating systems, load balancers, software patches, configuration tweaks, monitoring, and many other infrastructure details that must be ironed out.
With Azure, a completed application can be deployed in less than a day, including both a staging and production environment.
Reduce up-front capital expense by shifting to an operating expense model.
Systems with low-traffic periods really benefit from this model.
Hardware is a significant investment, especially when considering redundancy, load-balancing, and peak-capacity handling. This translates into a hefty up-front capital investment. Further, that investment might sit unused during low-traffic periods.
Azure provides an alternative, consumption-based model, with no capital expenses; only operating expenses. This results in a significant reduction in up-front investment. It also lets businesses focus on costs directly applicable to the application and related traffic. During down-time periods, Azure resources may be reduced or removed, resulting in additional cost savings.Built-in disaster recovery
Azure distributes an application’s virtual machines across separate server racks, as well as separate locations within a data center. This provides a 99.95% uptime Software Licensing Agreement (SLA), as no two servers are in the same fault-domain.
Going further, Azure backs up its SQL Azure relational databases and Windows Azure highly-scalable storage in a minimum of three locations, to provide a durable storage guarantee.
If an application’s hardware ever fails, the Azure fabric re-deploys an application’s affected instance immediately to new hardware.
Systems are automatically updated with the latest operating system patches
Microsoft provides operating system patches, as well as security-specific patches, for their server operating system products. Azure provides transparent maintenance with automatic operating system updates (these updates have an opt-out option).
During update rollouts, an application running a minimum of two VM instances will be updated in groups. The application will never be taken offline, as updates are only performed to one group at a time.Scale applications without any server licensing dependencies
As traffic increases, it’s critical to increase the number of running VM instances to provide adequate customer-facing performance. In a typical hosted environment, this means purchasing additional server licenses for both Windows Server 2008 and SQL Server 2008.
Azure, in its consumption-based model, has no additional charges for licensing. Simply select the number of running instances desired, and pay for those instances, starting at $0.12 per hour. Period.
Simplify SLA
With Azure providing the entire infrastructure and platform stack, including Internet, power, hardware, load-balancing, operating systems, deployment, management, redundancy, and upgrades, Microsoft provides a straightforward SLA for uptime and data availability.
If you are interested in getting started with Windows Azure, click here to learn about RDA's two-day Architecture/Design Session (ADS) for organizations interested in moving to cloud computing.
<Return to section navigation list>
JBarnes recommends that you Get Connected With News & Analysis From The Microsoft WPC 2010 in this 7/26/2010 post to the Innovation Showcase blog:
Peter Laudati and Dmitry Lyalin host the edu-taining Connected Show developer podcast on cloud computing and interoperability. Check out Episode #33, “Dmitry’s Soapbox”. Guest host Andrew Brust is back again, joining Dmitr Lyalin and Peter Laudati to talk about all of the tech news from Microsoft's Wordwide Partner Conference in Washington, DC. The trio talks about the new WebMatrix toolset and how it applies to both ASP.NET & PHP Developers.
Also on tap, news and analysis on Internet Explorer 9 Preview 3, Windows Phone 7, and the newly announced Windows Azure Appliance. A raging Dmitry also shares his poppin’ passion for HTML5. [Emphasis added.]
CLICK HERE TO LISTEN!
If you like what you hear, check out previous episodes of the Connected Show at www.connectedshow.com. You can subscribe on iTunes or Zune. New episodes approximately every two weeks!
Scott Wilson asks “what exactly is wrong with using your existing Microsoft-based datacenter?” in his Door Number Two post of 7/25/2010 to the CIO Weblog:
First, a question of semantics: is it insulting to describe anything to do with cloud computing as "vaporware?" Clouds being vaporous conglomerations themselves, it seems like we might need a new term to describe promised, but undelivered, cloud-based services.
I'm not sure that's what is going on with Microsoft's Azure right now, but just going by historical yardsticks, you have to wonder. The oh-so-mysterious Windows Azure Platform Appliance is "not for the faint of heart" writes Carl Brooks, but it's still pretty unclear who it is for. Very large enterprises, says Microsoft, but other than billing the device as a "proven cloud platform" it's not getting much more specific.
It sounds like a return to the "private cloud" schtick, which is its own sort of vaporware, if I may still use the term. Put a bunch of these bad boys in your datacenter, and you'll have Azure, only private. But if you're going to put a bunch of boxes in a datacenter and want to run the same stuff you can run on Azure, then what exactly is wrong with using your existing Microsoft-based datacenter?
Microsoft still hasn’t decided on a WAPA logo.
<Return to section navigation list>
Ewald Roodenrijs asserts “Cloud computing is an emerging phenomenon that offers enormous advantages” as a preface to his Mitigation of the Threats of the Cloud article of 7/26/2010:
Cloud computing is an emerging phenomenon that offers enormous advantages, such as shorter time to market, flexible computing capabilities and limitless power, but the cloud market, still in a very early stage, continues to grow and evolve.
As cloud computing evolves, it creates a global infrastructure for new possibilities used in software quality assurance and testing. Businesses can share public or hybrid clouds with each other or create private clouds to be shared within the whole company, instead of using separate options for different enterprise departments. However the cloud is also threatened by some risks. These risks should be addressed to create the highest result in implementing the cloud and avoid threats on the other hand.
Infrastructure requirements should be correct
With a flexible environment opportunity as the cloud the requirements of those environments should be made clear. When the requirements are not set correct or appropriate to what is really wanted, the result can be the direct opposite of what was the goal.When this happens a lot of negative noise will be generated about the possibilities of the cloud. This can result in a negative view on cloud computing itself. Cloud computing was at the top of the Gartner Hype Cycle of 2009 and a lot of negative points will show up in the media the coming months, a lot of these points will be the result of bad requirements around the infrastructure. Using a simple checklist this enables the opportunity to reduce this risk to a minimum.
Legacy systems can still be used
Almost all types of services and systems can be virtualized, even some legacy systems. But 5-10% of all systems cannot be virtualized, and most of these are legacy systems. Systems that are very important for the business, often the core business of a company makes use of these old mainframes for example. With using an interface with these legacy systems they can still be incorporated in the cloud. For example using a VPN connection between the cloud and the clients own servers can create a connection between the legacy systems and the cloud systems.Standardization and Virtualization
One of the first steps clients can take are test environments in the cloud. This is a short-term opportunity for solution integrators and is generating action from companies like IBM, CloudOne and various Telcos. The creation of standardized server and service models and the standardization of clients infrastructure has much more impact in the long term.For this cloud computing is a catalyst and a perfect excuse for IT modernization and to improve internal IT services maturity. It has an indirect impact on other infrastructure activities. Like application consolidation and portfolio rationalization, and therefore helping business figure out the cost of providing services internally and, subsequently, improving the efficiency and transparency of your IT operations.
Security
With all external forms of cloud computing data is transferred, processed and stored in an external (public) cloud. However, data owners are very skeptical to place their data outside their own control sphere. When (test) data is stored in the cloud this can lead to a compliancy issue for most businesses. The data owners these companies are responsible for the integrity and confidentiality of their data, even when the data is outside their direct control. Traditional solution integrators are forced to comply to external audits and obtain security certifications, so should cloud computing providers. But also the availability of data is inside their own control. Another company is responsible for the uptime of the servers. One of the cloud principles is to guarantee a very high uptime but this is still a threat they have to deal with.The most thorough security controls are needed to protect the most sensitive data. This may not be guaranteed in a public cloud, while it can be realized in a private cloud.
Transparency and traceability of (test) data
With cloud computing (test) data can be located in systems in other countries, which may be in conflict with regulations prohibiting data to leave a country or union. For example, the EU Data Protection Directive places restrictions on the export of personal data from the EU to countries whose data protection laws are not judges as "adequate" by EU standards (European Commission 1995a). If not properly attended to, European personal data may be located outside the EU without being compliant to the directive.Data segregation
The shared, massive scale characteristics of cloud computing makes it likely that clients data is stored alongside data of others consumers. Encryption is often used to segregate data-at-rest, but it is not a cure-all. It's advised to do a thorough evaluation of the encryption systems used by cloud providers.A proper built, but poorly managed encryption scheme may be just as devastating as no encryption at all, because although the confidentiality of data may be preserved, availability of data may be at risk when data availability is not guaranteed.
Cloud strategy
Currently cloud computing is a hype, but the cloud not only offers a lot of opportunities, but also threats. All these threats should be taken into account when creating a cloud strategy for your business!
Ewald is a senior test manager and a member of the business development team within Sogeti Netherlands.
James Staten asks How Much Infrastructure Integration Should You Allow? in this 7/25/2010 post to his Forrester blog:
There's an old adage that the worst running car in the neighborhood belongs to the auto mechanic. Why? Because they like to tinker with it. We as IT pros love building and tinkering with things, too, and at one point we all built our own PC and it probably ran about as well as the mechanic's car down the street.
While the mechanic's car never ran that well, it wasn't a reflection on the quality of his work on your car because he drew the line between what he can tinker with and what can sink him as a professional (well, most of the time). IT pros do the same thing. We try not to tinker with computers that will affect our clients or risk the service level agreement we have with them. Yet there is a tinkerer's mentality in all of us. This mentality is evidenced in our data centers where the desire to configure our own infrastructure and build out our own best of breed solutions has resulted in an overly complex mishmash of technologies, products and management tools. There's lots of history behind this mess and lots of good intentions, but nearly everyone wants a cleaner way forward.
In the vendors' minds, this way forward is clearly one that has more of their stuff inside and the latest thinking here is the new converged infrastructure solutions they are marketing, such as HP's BladeSystem Matrix and IBM's CloudBurst. Each of these products is the vendor's vision of a cleaner, more integrated and more efficient data center. And there's a lot of truth to this in what they have engineered. The big question is whether you should buy into this vision.
<Return to section navigation list>
CloudTweaks reported Skytap and Customer, Nuance Communications, to Speak on Cloud Computing at Burton Group (Gartner) Catalyst Conference based on a 7/26/2010 press release:
SEATTLE, WA–(Marketwire – July 26, 2010) – Skytap, Inc., the leading provider of self-service cloud automation solutions, today announced that Deanne Harper, senior manager of Speech University at Nuance Communications, and Sundar Raghavan, chief product and marketing officer at Skytap, will present at the Burton Group Catalyst Conference on July 28, 2010 in San Diego, CA.
At the event, Harper and Raghavan will provide real world insight into how the cloud can be used to accelerate business productivity, and share practical tips for companies to move to the cloud successfully. Sessions featuring Skytap and its customer, Nuance Communications, include:
Hands-on Training from the Cloud: Speech University’s Global Solution
Wednesday, July 28 at 4:05 PM
Speaker: Deanne Harper (Nuance Communications, Inc.)
Description: In this session, Deanne will describe the cloud-based training solution adopted by Nuance Speech University (NSU) in 2008. She will discuss the challenges that led NSU to consider the cloud as a training solution and the company’s requirements for a successful solution. Deanne will review her vendor comparison and identify factors that led to the solution adopted by Nuance.Cloud Economics and Licensing
Wednesday, July 28 at 3:10 PM
Panel Participants: Drue Reeves (Gartner), Sundar Raghavan (Skytap), Mario Olivarez (GoGrid), and Nathan Day (SoftLayer Technologies, Inc.)
Description: The roundtable will discuss the economics around the cloud and the myriad of licensing issues that also accompany a move to the cloud.Vendor Lightning Round
Wednesday, July 28 at 5:05 PM
Speakers: Sundar Raghavan (Skytap), Matt Tavis (Amazon.com), Nathan Day (SoftLayer Technologies, Inc.), and Jeff Samuels (GoGrid)
Description: In this session, cloud vendors present their vision and product in five minutes. The audience will vote to select a winner and the winner will receive 10 additional minutes to present.Catalyst is a five-day, industry-shaping conference exploring cutting-edge ideas, current challenges and emerging technologies shaping today’s and tomorrow’s enterprise. Catalyst Conference is renowned for its attendee-driven agenda, high-profile speakers, in-depth content and fiercely independent point of view.
The Burton Catalyst Conference will include 1.5 days of cloud-computing sessions on 2/27 and 2/29/2010 in San Diego, CA:
Session tracks include:
Networks in Motion Life After SOA: Next Generation Application Architecture Collaborate or Perish: The Business Value of Relationships The New Identity Architecture: Getting There from Here Security in Context: New Models for New Business Virtualization: Transforming IT Infrastructure Leveraging Information to Gain Insight Enterprise Ready Clouds: Realistic Strategies
Chris Hoff (@Beaker) asked See You At Black Hat 2010 & Defcon 18? in this 7/25/2010 post:
This year looks to be another swell get-together in Vegas. I had to miss last year (first time in…forever) so I’m looking forward to 112 degrees, recirculated air, and stumble-drunk hax0rs jackpotting ATMs and commandeering elevators.
I’ll be getting in on the 27th. I have a keynote at the Cloud Security Alliance Summit on the 28th (co-located within Black Hat,) a talk on the 29th at Black Hat (Cloudinomicon) from 10am-11am and I’ll be on another FAIL panel at Defcon with the boys. I’ve got a bunch of (gasp!) customer meetings and (gasp! x2) work stuff to do, but plenty of time for the usual.
I’m going to try to hit Cobra Kai, Xtreme Couture or the Tapout facilities whilst there for some no-gi grappling or even BJJ if I can find a class. Either way, there are some hard core P90X’ers that I’m sure I can con into working out in 90 degree, 6am weather.
Rumors of mojitos and cigars at Casa Fuente are completely unfounded. Completely.
Oh, parties? They have parties?
See y’all there!
/Hoff
Check out @Beaker’s Reflections on SANS ’99 New Orleans: Where It All Started reminiscences of the same date.
<Return to section navigation list>
Alex Handy’s CouchDB brings peer-based data replication article of 7/26/2010 for SDTimes explains CouchDB 1.0’s replication techniques:
Mobile devices and the move to the cloud have combined to form problems for which traditional SQL databases are unsuited. This is why the NoSQL movement has sprouted dozens of newly crafted databases for every imaginable problem.
For the Apache Foundation's CouchDB, replication and stability were the first priorities. On July 14, four years of work culminated in the release of Apache CouchDB 1.0, bringing with it the security features needed for enterprise applications.
Damien Katz created the CouchDB project after leaving the Lotus Notes team. He used his Notes knowledge to build a document database that could perform peer-based replication, along with a ground-up approach to tolerating node failure.
Version 1.0 puts the finishing touches on CouchDB's underpinnings, said Katz. Key to the purpose of the database is replication on a reliable and grand scale. Any node of a CouchDB cluster can be written to, and those changes will automatically trickle out to the other nodes, even if some of those other nodes are turned off at the time. And it doesn't matter how those nodes were turned off; Katz said that CouchDB is built to crash.
That's because the only way to turn off a CouchDB instance is with the Unix “kill” command. This is actually what the code does itself when a CouchDB instance is told to turn off. Katz claimed that, because CouchDB is designed to suddenly stop running, it can never corrupt the data it stores. It may sound unorthodox, but using a crash as the standard termination means there may be no way to surprise CouchDB.
“The replication stuff is the killer feature of CouchDB,” said Katz. “A lot of databases have some sort of replication capability, almost always master/slave, so reads can be spread across servers to reduce the read load. CouchDB uses peer-based replication, so any update can happen on any node and automatically replicate out. We have the ability to take a database offline so it's not connected to its other replicas, individually query it, then push that back to the replicas. It's fairly unique in the database world."
Alex reports “Katz has since founded Couchio, a company based in downtown Oakland and tasked with the creation of office productivity software on top of CouchDB.” [Emphasis and URL added.] You can learn more and sign up for a free cloud instance or download CouchDB at the Couchio web site:
Here’s an example of entering data into a new database and document using the Futon management tool:
Matthew Weinberger passed on to the MSPMentor blog on 7/26/2010 the LATimes report that Google Apps Misses Los Angeles Migration Deadline:
This is a case where the headline says it all. A Google Apps rollout at the City of Los Angeles has missed its migration deadline, the Los Angeles Times reports. The roadblock? The Los Angeles Police Department, which is refusing to move off the old system to Google’s SaaS-based approach. Here’s why.
Let’s take a step back. When Google and Los Angeles finalized the high-profile deal back in December 2009, both sides said the deal aimed at helping the municipal government save on infrastructure costs. But MSPmentor noted that privacy concerns could hold the project back.
That turned out to be prophetic. May 2010 brought rumors that the LAPD had security and compliance concerns that were holding up the project. Google itself dispelled those rumors for us the next day, and migration appeared to be proceeding apace.
Fast forward to July 2010: The LA Times story says Google missed the June 30th deadline, leaving almost 20,000 city workers on an aging Novell legacy platform and forcing the city to pay licenses for both systems — at a cost that could reach as much as $400,000 over the next year. The cause? The Los Angeles Police Department and their dissatisfaction with the SaaS platform’s security.
“Google executive Jocelyn Ding said the company was committed to fulfilling its contract but admitted that it had missed ‘some details’ in the original requirements,” writes the LA Times.
Naturally, no one’s happy: the City of Los Angeles is left scratching its head as to how their great bargain didn’t encompass LAPD requirements, and Google’s committed themselves to picking up the tab on the legacy system until at least November, according to the report.
The City of Los Angeles engagement was supposed to be Google’s poster child for SaaS email replacing more traditional on-premises email systems from Microsoft, IBM Lotus and Novell. Google seems serious about the enterprise space, but we’re curious to see how other potential Google Apps customers react to the City of Los Angeles situation.
Google doesn’t appear to have commented on the matter publicly beyond the above quote, and the official Los Angeles migration project blog has been silent since June 23rd, 2010. Needless to say, MSPmentor is keeping its ears open for further developments.
Read More About This Topic
It’s ironic that ebizQ reported on the same day that Google Announces Google Apps for Government and claims to have received FISMA certification for them:
This was posted today on the Google blog: Today we’re excited to announce a new edition of Google Apps. Designed with guidance from customers like the federal government, the City of Los Angeles and the City of Orlando, Google Apps for Government includes the same great Google applications that people know and love, with specific measures to address the policy and security needs of the public sector.
Take Berkeley Lab, a member of the national laboratory system supported by the U.S. Department of Energy. It’s managed by the University of California and conducts unclassified research across a wide range of scientific disciplines. Berkeley researchers collaborate with scientists around the world, so emailing version upon version of documents among collaborators and trying to juggle disparate files is difficult. Berkeley Lab researchers have been using Google Apps to share documents that live in the cloud, and can view and edit documents and spreadsheets simultaneously knowing they are always working from the latest information. (Read more from Berkeley Lab’s Chief Information Officer on the Enterprise blog.)
And we’re not stopping with FISMA certification. Google Apps for Government will continue to evolve to meet unique government requirements. Google Apps for Government stores Gmail and Calendar data in a segregated system located in the continental United States, exclusively for our government customers. Other applications will follow in the near future. The suite is a “community cloud”—as defined by the National Institute for Science and Technology—to support the needs of our government customers. Google Apps for Government is available now to any federal, state or local government in the United States.
With reviews of our security controls in place, government agencies can more easily take advantage of all the benefits of one of the world’s best cloud computing systems. Google’s cloud offers higher reliability, best-in-class disaster recovery and access to a steady stream of innovation—all of which can provide substantial improvements over existing systems in addition to significant cost savings. And with no hardware or software to install and maintain, Google Apps for Government allows agencies to redeploy resources to technology projects core to their mission of serving the public. This new edition should give governments an even stronger case for making the move to the cloud.
FISMA obviously doesn’t require meeting migration deadlines. However, I wonder why the LAPD wasn’t satisfied with FISMA security certification if security and compliance were the sore spots.
Update: Sam Diaz explains the LAPD’s “sore spots” in a later Google's hiccups in L.A. highlight need for more education about cloud apps post of 7/26/2010 to ZDNet’s Between the Lines blog:
Today’s Google Apps for Government news comes days after Google and the city of Los Angeles made headlines about some speed bumps in the rollout of Google Apps for the city.
At the press event today, execs took a few minutes for questions about the Los Angeles rollout and said that, in part, the federal FISMA certification has helped to address some of the city’s concerns. A thumbs-up from FISMA is, in a sense, working as a blanket “banner of trust” that state and local governments can point to for reassurances.
One of the concerns expressed by the city of Los Angeles - specifically its police department - dealt with background checks on Google employees with access to LAPD data. Google said its employees already go through background checks to ensure the security of data on its clouds but that Los Angeles wanted checks specific to its data.
The rollout with the city of Los Angeles is a unique one because 1) it’s the second largest city in the U.S. and 2) it is one of the first of its kind to jump into the cloud. To a certain extent, that means both the city and Google are in uncharted territory.
It also means there’s a learning curve that comes with the rollout - and Google said one of the things it maybe could have done better is to explain how Google’s process works.
In the past, companies - and government agencies - could actually see their physical servers, as well as the people in charge of managing and securing them. Literally, execs could look those employees in the eye - and that brings a certain amount of peace of mind.
Handing over keys to the data to a third party company is scary enough but not knowing who, within that third-party company, actually has access can be downright frightening. The Google execs said they understood that and are working with customers to address those concerns early.
Having the federal certification makes it easier, they said.
William Vambenepe (@vambenepe) continues the cloud-standards fracas with his updated The Tragedy of the Commons in Cloud standards of 7/25/2010 (make sure to read the update at the end of the post):
I wasn’t at the OSCON Cloud Summit this past week, but I’ve spent some time over the weekend trying to collect the good bits. Via Twitter, I had heard echos of an interesting debate on Cloud standards between Sam Johnston and Benjamin Black. Today I got to see Benjamin’s slides and read reports from two audience members, Charles Engelke and Krishnan Subramanian. Sam argued that Cloud standards are needed, Benjamin that they would be premature.
Benjamin is right about what to think and Sam is right about what to do.
Let me put it differently: Benjamin is right in theory, but it doesn’t matter. Here is why.
Say I’m a vendor and Benjamin convinces me
Assume I truly believe the industry would be better served if we all waited. Does this mean I’ll stay away from Cloud standards efforts for now? Not necessarily, because nothing is stopping my competitors from doing it. In the IT standards world, your only choice is to participate or opt out. For the most part you can’t put your muscle towards stopping an effort. Case in point, Amazon has so far chosen to opt out; has that stopped VMWare and others from going to DMTF and elsewhere to ratify specifications as standards? Of course not. To the contrary, it has made the option even more attractive because when the leader stays home it is a lot easier for less popular candidates to win the prize. So as a vendor-who-was-convinced-by-Benjamin I now have the choice between letting my competitor get his specification rubberstamped (and then hit me with the competitive advantage of being “standard compliant” and even “the standard leader”) or getting involved in an effort that I know to be counterproductive for the industry. Guess what most will choose?
Even the initial sinner (who sets the wheels of premature standardization in motion) may himself be convinced that it’s too early for Cloud standards. But he has to assume that one of his competitors will make the move, and in that context why give them first mover advantage (and the choice of the battlefield). It’s the typical Tragedy of the Commons scenario. By acting in a rational and self-interested way, participants invariably end up creating a bad situation, one that they might all know is against everyone’s self interest.
And it’s not just vendors.
Say I’m an officer of a Standard-setting organization and Benjamin convinces me
If you expect that I would use my position in the organization to prevent companies from starting a Cloud standard effort there, you live in fantasy-land. Standard-setting organizations compete with one another just as fiercely as companies do. If I have achieved a position of leadership in a given standard organization, the last thing I want is to see another organization lay claims to a strategic and fast-growing area of the IT landscape. It takes a lot of time and money for a company to get elected on the right board and gets its employees (or other reliable allies) in the right leadership positions. Or to acquire people already in that place. You only get a return on that investment if the organization manages to be the one where the key standards get created. That’s what’s behind the landgrab reflex of many standards organizations.
And it goes beyond vendors and standards organizations
Say I’m an IT buyer and Benjamin convinces me
Assume I really believe Cloud standards are premature. Assume they get created anyway and I have to choose between a vendor who supports them and one who doesn’t. Do I, as a matter of principle, refuse consider the “standard-compliant” label in my purchasing decision? Even if I know that the standard shouldn’t have been created, I also know that, all other things being equal, the “standard-compliant” product will attract more tools and complementary solutions and will likely ease future integration problems.
And then there is the question of how I’ll explain this to my boss. Will Benjamin be by my side with his beautiful slides when I am called in an emergency meeting to explain to the CIO why we, unlike the competitors, didn’t pick “a standards-based solution”?
In the real world, the only way to solve problems caused by the Tragedy of the Commons is to have some overarching authority regulate the usage of the resource at risk of being ruined. This seems unlikely to be a workable solution when the resource is not a river to protect from sewer discharges but an IT domain to protect from premature standardization. If called, I’d be happy to serve as benevolent dictator for the IT industry (I could fix a few other things beyond the Cloud standards landgrab issue). But as long as neither I nor anyone else is in a dictatorial position, Benjamin’s excellent exposé has no audience for which his call to arms (or rather to lay down the arms) is actionable. I am not saying that everyone agrees with Benjamin, but that even if everyone did it still wouldn’t make a difference. Many of us in the industry share his views and rationally act as if we didn’t.
[UPDATED 2010/7/25: In a nice example of Blog/Twitter synergy, minutes after posting this I was having a conversation on Twitter with Benjamin Black about my interpretation of what he said. Based on this conversation, I realize that I should clarify that what I mean by "standards" in this post is "something that comes out of a standard-setting organization" (whether or not it gets adopted), in other words what Benjamin calls a "standard specification". He uses the word "standard" to mean "what most people use", which may or may not be a "standard specification". That's a big part of the disconnect that led to our Twitter chat. The other part is that what I presented as Benjamin's thesis in my post is actually only one of the propositions in his talk, and not even the main one. It's the proposition that it is damaging for the industry when a standard specification comes out of a standard organization too early. I wasn't at the conference where Benjamin presented but it's hard to understand anything else out of slide 61 ("standardize too soon, and you lock to the wrong thing") and 87 ("to discover the right standards, we must eschew standards"). So if I misrepresented him I believe it was in making it look like this was the focus of his talk while in fact it was only one of the points he made. As he himself clarified for me: "My _actual_ argument is that it doesn't matter what we think about cloud standards, if they are needed, they will emerge" (again, in this sentence he uses "standards" to mean "something that people have converged on").
More generally, my main point here has nothing to do with Benjamin, Sam and their OSCON debate, other than the fact that reading about it prompted me to type this blog entry. It's simply that there is a perversion in the IT standards landscape that makes it impossible for premature standardization *not* to happen. It's something I've written before, e.g. in this post:
Saying “it’s too early” in the standards world is the same as saying nothing. It puts you out of the game and has no other effect. Amazon, the clear leader in the space, has taken just this position. How has this been understood? Simply as “well I guess we’ll do it without them”. It’s sad, but all it takes is one significant (but not necessarily leader) company trying to capitalize on some market influence to force the standards train to leave the station. And it’s a hard decision for others to not engage the pursuit at that point. In the same way that it only takes one bellicose country among pacifists to start a war.
Benjamin is just a messenger; and I wasn't trying to shoot him.]
Related posts:
William works as architect for the application and middleware management part of Oracle Enterprise Manager.
Nancy Gohring posits “The ClearPath cloud offering will be initially geared toward test and development applications” in a preface to her Unisys to offer mainframe cloud services article of 7/23/2010 for InfoWorld’s Cloud Computing blog:
Unisys plans to offer a hosted test and development environment later this year for users of its ClearPath mainframe systems, a precursor to offering hosted production environments in the future, the company said this week.
The initial offering is geared toward businesses that already have a ClearPath test and dev environment, and who need to expand that environment for a brief period of time.
Unisys plans to offer a hosted test and development environment later this year for users of its ClearPath mainframe systems, a precursor to offering hosted production environments in the future, the company said this week.
The initial offering is geared toward businesses that already have a ClearPath test and dev environment, and who need to expand that environment for a brief period of time.
"We are trying to address the issue of agility, because customers have requirements for short bursts of resources for development and test. What this allows them to do is avoid the cost if they had to buy another system," Bill Maclean, vice president of ClearPath portfolio management at Unisys, said on Friday.
ClearPath users tend to be large organizations such as airlines, telecom operators or financial institutions. They are likely to be interested in the cloud offering to test new software or catch up on projects that they have fallen behind on, he said.
Partly due to the nature of the offering, the service comes with an unusual pricing model. While most platform- or infrastructure-as-a-service offerings are priced based on CPUs and memory, the ClearPath Cloud Development and Test Solution has a set price for use over a three-month period.
"We license it for a period of time and they get the full capability. It's like a full virtual server with a complete set of development technologies," he said. The entry-level price, $13,000 for three months, includes a full set of development tools such as compilers, editors and database software. It also includes memory, storage and support.
Pricing it like the rest of the infrastructure-as-a-service industry "wouldn't provide a whole lot of benefit because in the ClearPath environment, the software comes with the platform," Maclean said.
Unisys will offer additional pricing tiers for more CPU resources, although Maclean expects that most will take the entry level since it's large.
The company decided to offer the service to customers after using it internally for about two years. It set up an on-demand service for Unisys engineers who needed quick access to compute power. "We did it internally because we had a business problem," he said. The company realized that if it had this problem, its customers probably did too.
Unisys expects to add other ClearPath cloud services in the future, including a hosted production environment. That might make sense for a small customer that wants to host its entire environment at Unisys, or for larger companies that want to host a specific application. It also expects to add data replication and recovery services, and select application-as-a-service offerings.
The test and development offering is expected to be available worldwide in September. Unisys will be hosting it out of its data center in Eagan, Minnesota.
<Return to section navigation list>
by --rj (roger_jennings@compuserve.com) at July 26, 2010 05:36 PM
When strategies are formed it quickly becomes obvious that cloud computing is more about balance than anything else.
At a time when you’d think cloud computing would be the primary “go to” strategy for managing scale and rapid growth multiple well-known and demanding organizations are building their own data centers instead.
With all the hype around cloud being faster, cheaper, and more efficient these folks must be crazy, right?
Not at all. In fact, these moves illustrate the growing friction between the economy of scale offered by cloud computing and the control and flexibility that is part and parcel of owning one’s own data center.
“Later this year, Twitter is moving our technical operations infrastructure into a new, custom-built data center in the Salt Lake City area,” wrote Twitter’s Jean-Paul Cozzatti, who said having dedicated data centers will provide more capacity to accommodate growth of 300,000 new users per day. “Keeping pace with these users and their Twitter activity presents some unique and complex engineering challenges. Importantly, having our own data center will give us the flexibility to more quickly make adjustments as our infrastructure needs change.”
-- Data Center Knowledge, “Twitter Picks Utah for New Data Center”
Twitter isn’t the only Web 2.0 savvy organization moving to their own data center. Facebook earlier this year announced it, too, was also investing in building out its own data center.
It’s not all about scalability. I know that sounds nearly heretical, but it’s not. And it’s not a new mantra, either. Scalability is certainly a factor in why one would choose cloud computing over a localized deployment, but also important are control and flexibility.
Another consideration is the ability to customize your data center infrastructure to provide more granular control of operations. “That control gives us a ton of flexibility, and we can build new things without having to wait for our partner,” said Heiliger [Jonathan Heiliger, Facebook’s VP of Technical Operations]
-- Data Center Knowledge, “Data Centers: For When The Cloud is Not Enough”
If I’ve said it once I’ve said it a thousand times: control is a huge factor in the decision making process and 
something that isn’t effectively offered by today’s public cloud computing offerings. Remember the Information week analytics Cloud computing survey in 2009?
Even though security remains concern number one, control and configurability are on the top of the list, as well. The issue of control has almost always gone hand in hand with cloud adoption inhibitors, but it always takes a back seat to the more glamorous and scary “security” issue. These are not minor stumbling blocks in many cases, and the inability to rapidly adapt an infrastructure to meet growth and scale and make architectural changes, if necessary, are paramount to success. If cloud computing cannot provide the agility necessary to meet these challenges then it is logical to assume that organizations will either (a) stay in the local data center or (b) move to a local data center from the cloud when it becomes obvious the environment is inhibiting forward momentum.
Current adoption patterns indicate that this is not an anomaly, but will instead likely become the norm for organizations. Applications that are initially deployed “in the cloud” will, upon becoming a critical business application or growing beyond the meager means of control and flexibility offered by the cloud, will 
migrate to the data center, where control and agility are provided by the simple fact that the organization can change at will any piece of the infrastructure – from its physical implementation to its logical organization – at will. This is evident in the percentage of organizations using cloud for “dev and test” but not for production. Clearly the economy of scale and rapidity of deployment makes the cloud a perfect environment for development and testing but not necessarily production.
At some point scalability can become not about the application but about its infrastructure and the way in which that infrastructure interacts. It can become about the network and its components and how applications end up interacting with and through that infrastructure. In a cloud computing environment it is rarely the case that a customer can impact that infrastructure and, when it can, it is then limited by other factors such as underlying virtualization technology and the physical server infrastructure on which the application is ultimately deployed. If the answer to a scalability obstacle is more bandwidth and higher throughput, you can’t really add another NIC to a server in the cloud. That’s not your call. But it is if you’re in the data center, and it is virtualization – not cloud - that ultimately provides the agility to make such a change and rapidly propagate that change across the application deployment.
It isn’t always about costs. Well, okay, it is about cost but in IT it’s about cost as it relates to performance, or flexibility, or other operational functionality required to successfully meet data center and business goals. When spending less on infrastructure results in higher operational costs, the organization really hasn’t saved money at all. Savvy CIO and CTOs understand that it’s not a battle, but a balancing act. It’s not about achieving the highest economy of scale, but the best economy of scale given the specific operational and business needs.
Related Postsfrom tag control |
|
| A compendium of Windows Azure, Windows Azure Platform Appliance, SQL Azure Database, AppFabric and other cloud-computing articles. |
• Update 7/25/2010: Noted more content in the Bruno Terkaly continues his Leverage Cloud Computing with Windows Azure and Windows Phone 7 – Step 2 to Infinity – Under Construction series post in the Live Windows Azure Apps, APIs, Tools and Test Harnesses section.
Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:
To use the above links, first click the post’s title to display the single article you want to navigate.
Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)
Read the detailed TOC here (PDF) and download the sample code here.
Discuss the book on its WROX P2P Forum.
See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.
Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.
You can now freely download by FTP and save the following two online-only PDF chapters of Cloud Computing with the Windows Azure Platform, which have been updated for SQL Azure’s January 4, 2010 commercial release:
HTTP downloads of the two chapters are available for download at no charge from the book's Code Download page.
Tobin Titus (@tobint) explains the CloudStorageAccount.SetConfigurationSettingPublisher and CloudStorageAccount.FromConfigurationSetting methods in this 7/23/2010 post:
Those venturing into Windows Azure development for the first time may find themselves in a bit of a quandary when they try to devise a strategy for getting their storage account credentials. You dive into the Windows Azure APIs and determine that you must first get a reference to a CloudStorageAccount. You are all aglow when you find the “FromConfigurationSetting” method that takes a single string parameter named “settingName”. You assume that you can store your connection information in a configuration file, and call “FromConfigurationSetting” from your code to get dynamically load the connection information from the config file. However, like most of the Windows Azure documentation, it is very light on details and very heavy on profundity. The documentation for “FromConfigurationSetting” currently says:
“Create a new instance of a CloudStorageAccount from a configuration setting.”
That’s almost as helpful as “The Color property changes the color.” Notice that it isn’t specific about what configuration setting this comes from? The documentation is purposefully generic, but falls short of explaining why. So I’ve seen multiple questions in forums and blogs trying to figure out how this works. I’ve seen developers try to put the configuration setting in various places and make assumptions that simply providing the configuration path would get the desired result. Then they try to run the code and they get the following exception message:
“InvalidOperationException was caught
SetConfigurationSettingPublisher needs to be called before FromConfigurationSetting can be used”
Of course, this causes you to look up “SetConfigurationSettingPublisher” which also is a little light on details. It says:
“Sets the global configuration setting publisher.”
Which isn’t entirely clear. The remarks do add some details which are helpful, but don’t go as far as to help you understand what needs to be done – hence this blog post. So that was the long setup to a short answer.
The FromConfigurationSetting method makes no assumptions about where you are going to store your connection information. SetConfigurationSettingPublisher is your opportunity to determine how to acquire your Windows Azure connection information and from what source. The FromConfigurationSetting method executes the delegate passed to SetConfigurationSettingPublisher – which is why the error message says this method needs to be called first. The delegate that you configure should contain the logic specific to your application to retrieve the storage credentials. So essentially this is what happens:
- You call SetConfigurationSettingPublisher, passing in the logic to get connection string data from your custom source.
- You call FromConfigurationSetting, passing in a setting name that your delegate can use to differentiate from other configuration values.
- FromConfigurationSetting executes your delegate and sets up your environment to allow you to create valid CloudStorageAccount instances.
So, going back to a typical application, you want to store your configuration in a .config file and retrieve it at runtime when your Azure application runs. Here’s how you’d set that up. When your application starts up, you’ll want to configure your delegate using SetConfigurationSettingPublisher as follows:
If this is an ASP.NET application, you could call this in your Application_Start event in your global.asax.cs file. If this is a desktop application, you can set this in a static Constructor for your class or in your application startup code.
In this example, you are getting a configuration string the Windows Azure configuration file, and using the delegate to assign that string to the configuration setting publisher. This is great if you are executing in Windows Azure context all the time. But if you aren’t running your code in the Windows Azure context (for instance, you are exercising your API from NUnit tests locally) this example will throw another exception:
“SEHException was caught
External component has thrown an exception.”
To solve this problem, you can set up the delegate to handle as many different environments as you wish. For example, the following code would handle two cases – inside a Windows Azure role instance, and in a local executable:
For your unit tests to work, just add an appSettings key to your configuration file:
Note: Make sure to replace <accountName> and <accountKey> with your custom values.
In the context of your Windows Azure application, just make sure that you configure the same configuration key name there. In Visual Studio:
- Right click your Windows Azure role and choose Properties from the dropdown menu.
- Click on the Settings tab on the left.
- Click the Add Setting button in the button bar for the Settings tab.
- Type “DataConnectionString” in the Name column
- Change the Type drop down box to Connection String
- Click the “…” (ellipsis) button in the right hand side of the Value column
- Configure your storage credentials using the values from the Windows Azure portal
- Click OK to the dialog
- Use the File | Save menu option to save your settings then close the settings tab in visual studio.
With this code, you should now be able to retrieve your connection string from configuration settings in a Windows Azure role and in a local application and retrieve a valid CloudStorageAccount instance using the following code:
Of course, you aren’t limited to these scenarios alone. Your SetConfigurationSettingPublisher delegate may retrieve your storage credentials from anywhere you choose – a database, a user prompt, etc. The choice is yours.
For those of you who need a quick and dirty way to get an CloudStorageAccount instance without all the hoops, you can simply pass your connection string in to the Parse or TryParse method.
This should give you all the information you need to get a CloudStorageAccount instance and start accessing your Windows Azure storage.
Tobin is a member of Microsoft’s Internet Explorer team.
<Return to section navigation list>
Wayne Walter Berry (@WayneBerry) continues his PowerPivot for the DBA: Part 3 series on 7/23/2010:
In this article I am going to continue to tie some simple terminology and methodology in business intelligence with Transact-SQL – bring it down to earth for the DBA. This is a continuation in a series of blog posts (part 1, part 2) that equates Transact-SQL to PowerPivot.
Scope
As discussed in this previous blog post, a measure is passed the rows of the cell it is evaluating, the scope of the cell in the PowerPivot table. Which works really well if you are summing a single cell in that scope. However, what if you want to get a ratio between this scope and a larger scope, like that of the row the cell is in, or the whole PivotTable?
Measures have the ability to reach outside of their scope and draw in information from the bigger picture. If a cell is the PivotTable is a set of rows that are in the Bike category and the have an order date of 7/1/2001 the measure has access to all the rows that are in the Bike Category, and all the rows with an order date of 7/1/2001, it even has access to all the rows in the whole PivotTable.
The ability to access more than just the local scope is the power of the measure in PowerPivot. This feature gives the user insight into to how the cell data compares to other tables in the PivotTable.
ALL
In Data Analysis Expressions (DAX), the language used in the measure formula, ALL returns all the rows in a table, or all the values in a column, ignoring any filters that might have been applied. Here is an example of using ALL to calculate the ratio of total line item sales in the cell to the total line item sales for the category:
=SUM(SalesOrderDetail[LineTotal])/CALCULATE(SUM(SalesOrderDetail[LineTotal]),ALL(ProductCategory))Find the division in the formula, to the left is a formula that we already discussed in this previous blog post; it sums all the LineTotal columns in the scope of the cell. To the right of the division is the interesting part of the formula, it invokes the CALCULATE keyword to change the scope of the summation. CALCULATE evaluates an expression in a context that is modified by the specified filters. In this case those filters are all the rows returned from the result in this particular row in the PivotTable. In the example above this is the order date. Here is what the results look like:
Transact-SQL
Now let’s get the same results with Transact-SQL, this turns into a 200 level transact-SQL statement, because of the nested SELECT used as a table (T1) to get the summation of the LineTotal column per date.
SELECT ProductCategory.Name, SalesOrderHeader.OrderDate, SUM(LineTotal)/ MAX(T1.ProductCategoryTotal) FROM Sales.SalesOrderHeader INNER JOIN Sales.SalesOrderDetail ON SalesOrderHeader.SalesOrderID = SalesOrderDetail.SalesOrderID INNER JOIN Production.Product ON Product.ProductID = SalesOrderDetail.ProductID INNER JOIN Production.ProductSubcategory ON Product.ProductSubcategoryID = ProductSubcategory.ProductSubcategoryID INNER JOIN Production.ProductCategory ON ProductSubcategory.ProductCategoryID = ProductCategory.ProductCategoryID INNER JOIN ( SELECT SUM(LineTotal) 'ProductCategoryTotal', SalesOrderHeader.OrderDate FROM Sales.SalesOrderHeader INNER JOIN Sales.SalesOrderDetail ON SalesOrderHeader.SalesOrderID = SalesOrderDetail.SalesOrderID GROUP BY SalesOrderHeader.OrderDate ) AS T1 ON SalesOrderHeader.OrderDate = T1.OrderDate GROUP BY ProductCategory.Name, SalesOrderHeader.OrderDate ORDER BY SalesOrderHeader.OrderDateThis returns all the right results, however it isn’t very pretty compared to the PivotTable in Excel. The percents are not formatted, the results are not pivoted, there are no grand totals and the data isn’t very easy to read.
Here is the Transact-SQL to pivot the table:
SELECT OrderDate, [1] AS Bikes, [2] AS Components, [3] AS Clothing, [4] AS Accessories FROM (SELECT Sales.SalesOrderDetail.LineTotal/T1.ProductCategoryTotal 'LineTotal', ProductCategory.ProductCategoryID, SalesOrderHeader.OrderDate FROM Sales.SalesOrderHeader INNER JOIN Sales.SalesOrderDetail ON SalesOrderHeader.SalesOrderID = SalesOrderDetail.SalesOrderID INNER JOIN Production.Product ON Product.ProductID = SalesOrderDetail.ProductID INNER JOIN Production.ProductSubcategory ON Product.ProductSubcategoryID = ProductSubcategory.ProductSubcategoryID INNER JOIN Production.ProductCategory ON ProductSubcategory.ProductCategoryID = ProductCategory.ProductCategoryID INNER JOIN ( SELECT SUM(LineTotal) 'ProductCategoryTotal', SalesOrderHeader.OrderDate FROM Sales.SalesOrderHeader INNER JOIN Sales.SalesOrderDetail ON SalesOrderHeader.SalesOrderID = SalesOrderDetail.SalesOrderID GROUP BY SalesOrderHeader.OrderDate ) AS T1 ON SalesOrderHeader.OrderDate = T1.OrderDate ) p PIVOT ( SUM(LineTotal) FOR ProductCategoryID IN ( [1], [2], [3], [4] ) ) AS pvt ORDER BY pvt.OrderDate;
Maarten Balliauw’s (@maartenballiauw) Manage your SQL Azure database from your browser post of 7/23/2010 is an abbreviated version of my Test Drive Project “Houston” CTP1 with SQL Azure updated on 7/23/2010:
Yesterday, I noticed on Twitter that the SQL Azure - Project “Houston” CTP 1 has been released online. For those who do not know Houston, this is a lightweight and easy to use database management tool for SQL Azure databases built in Silverlight. Translation: you can now easily manage your SQL Azure database using any browser. It’s not a replacement for SSMS, but it’s a viable, quick solution into connecting to your cloudy database.
A quick look around
After connecting to your SQL Azure database through http://manage.sqlazurelabs.com, you’ll see a quick overview of your database elements (tables, views, stored procedures) as well as a fancy, three-dimensional cube displaying your database details.
Let’s create a new table… After clicking the “New table” toolbar item on top, a simple table designer pops up:
You can now easily design a table (in a limited fashion), click the “Save” button and go enter some data:
Stored procedures? Those are also supported:
Even running stored procedures:
Conclusion
As you can probably see from the screenshots, project “Houston” is currently quite limited. Basic operations are supported, but for example dropping a table should be done using a custom, hand-crafted query instead of a simple box.
What I would love to see is that the tool gets a bit more of the basic database operations and a Windows Phone 7 port? That would allow me to quickly do some trivial SQL Azure tasks both from my browser as well as from my (future :-)) smartphone.
Martin’s initial “It’s not a replacement for SSMS, but it’s a viable, quick solution into connecting to your cloudy database” comment is similar to my concluson.
Richard Seroter’s Using “Houston” to Manage SQL Azure Databases post of 7/22/2010 contains yet another guided tour of SQL Houston which begins:
First, you need to create a SQL Azure database (assuming that you don’t already have one). Mine is named SeroterSample. I’m feeling very inspired this evening.
Next up, we make sure to have a firewall rule allowing Microsoft services to access the database.
After this, we want to grab our database connection details via the button at the bottom of the Databases view.
Now go to the SQL Azure labs site and select the Project Houston CTP 1 tab.
We then see a futuristic console which either logs me into project Houston or launches a missile. …
Richard continues with a demo similar to Maarten’s and mine.
Pinal Dave’s SQLAuthority News – Guest Post – Walkthrough on Creating WCF Data Service (OData) and Consuming in Windows 7 Mobile application post of 7/23/2010 begins:
Objective
In this article, I will discuss:
- How to create WCF Data Service
- How to remove digital signature on System.Data.Service.Client and add in Windows7 phone application.
- Consume in Windows 7 phone application and display data.
You can see three videos here:
- Creating WCF Data Service: http://dhananjaykumar.net/2010/06/08/wcfdataservicevideo/
- Fixing the Bug of OData Client library for Windows7 Phone: http://dhananjaykumar.net/2010/07/01/videofixingodataclientlibraryissue/
- Consuming WCF Data Service in Windows7 Phone: http://dhananjaykumar.net/2010/07/06/videodatainw7phone/
Dhananjay continues with a detailed step-by-step tutorial for consuming OData information in a Windows Phone 7 application.
The OData team reported the availability of an OData Client for Objective-C for iPhone, iPad and MacOSX in this 7/22/2010 post:
The source code has been made available under the Apache 2.0 license and is available for download at http://odataobjc.codeplex.com
Liam Cavanagh (@liamca) announced that all registrants will be granted immediate access to SQL Server Data Sync in his Data Sync Service – Registration post of 7/22/2010:
Today we have opened up access to the Data Sync Service for SQL Azure CTP to all users. Previously, users who wanted to use the Data Sync Service to synchronize their SQL Azure databases were asked to register for access using a Live ID from which we slowly added users to the service in a controlled fashion. I am happy to announce that we have now approved all of these requests and now any users who register will be automatically approved.
To get started please visit: http://sqlazurelabs.com and click on "SQL Azure Data Sync"
<Return to section navigation list>
Ron Jacobs reported the availability of this 00:30:05 endpoint.tv - Workflow and Custom Activities - Best Practices (Part 4) video segment on 7/23/2010:
In this episode, Windows Workflow Foundation team Program Manager Leon Welicki drops in to show us the team's guidelines for developing custom activities.
In Part 4, we cover more Activity Design guidelines including
- Body vs. Children
- Variables
- Overriding CacheMetadata
- Activity Design
For more information
<Return to section navigation list>
• Bruno Terkaly continues his Leverage Cloud Computing with Windows Azure and Windows Phone 7 – Step 2 to Infinity – Under Construction series on 7/23/2010 with this brief post (updated 7/25/2010 for added content):
Building the Infrastructure
Our code will be composed of many projects
In order for a phone application to talk to the cloud to get data, it needs a few things setup. Here are the things that we are going to build:
- A web service that hosts a SL-enabled WCF Service
- A Silverlight client for both browser[s] and Windows Phone[s] 7
- A data tier to host our data
- A SQL Azure database and a SQL Server On-Premise database
Bruno continues with 10 similarly-illustrated steps and concludes:
More posts coming soon
Building the Windows Azure and Windows Phone 7
One step at a time is how I roll. So stay tuned for more detailed steps. Keep the emails flowing and let me know what works and what doesn’t.
“Soma” Somasegar’s MSDN: Double the Azure post of 7/23/2010 confirms yesterday’s report by Eric Golpe:
Windows Azure is a is a flexible cloud-computing platform that provides developers with on-demand compute and storage to host, scale, and manage web applications on the internet through Microsoft datacenters.
This January we introduced Windows Azure benefits as part of the MSDN Premium, Ultimate and BizSpark subscriptions with an eight month introductory offer. This offer allows MSDN subscribers to take advantage of the benefits of the Windows Azure platform, including the ability to quickly scale up or down based on your business need without the hassle of dealing with operational hurdles such as server procurement, configuration, and maintenance. With Azure, you pay only for what you use.
I encourage MSDN subscribers and BizSpark members to sign up for their Azure benefits if you have not done so already.
More details on this can be found on the Windows Azure Platform Benefits for MSDN Subscribers page.
Apparently, the Windows Azure team jumped the gun by updating the MSDN Subscribers page yesterday.
Jim O’Neil continues his @home With Windows Azure series with Azure@home Part 2: WebRole Implementation of 7/23/2010:
This post is part of a series diving into the implementation of the @home With Windows Azure project, which formed the basis of a webcast series by Developer Evangelists Brian Hitney and Jim O’Neil. Be sure to read the introductory post for the context of this and subsequent articles in the series.
In my last post, I gave an overview of the @home With Windows Azure architecture, so at this point we’re ready to dive in to the details. I’m going to start with the internet-facing WebRole, the user interface for this application, and cover it in two blog posts, primarily because we’ll broach the subject of Azure storage, which really deserves a blog post (or more) in and of itself.
Getting to Where You Already Are
By now, you should have downloaded the relatively complete Azure@home application in either Visual Studio 2008 or 2010 format and magically had a bunch of code dropped in your lap – a cloud project, a web role, a worker role and a few other projects thrown in the mix. Before we tackle the WebRole code in this article, I wanted to quickly walk through how the solution was built-up from scratch. If you’ve built Azure applications before, even “Hello World”, feel free to skip this section (not that I’d really know if you did or didn’t!)
You start off just as you would to create any other type of project in Visual Studio: File->New->Project (or Ctrl+Shift+N for the keyboard junkies out there). Presuming you have the Azure Tools for Visual Studio installed, you should see a Cloud option under the Visual C# and Visual Basic categories. (It doesn’t show under Visual F# even though you can build worker roles with F#).
In the New Project dialog (above), there’s not much to choose from; however, this is where you’ll need to commit to running either .NET Framework 3.5 or .NET Framework 4 inside of Windows Azure. At the point when we started this project, there wasn’t any .NET 4 support in the cloud, so the project we’ll be working with is still built on the 3.5 framework. I generally elect to build a new solution as well when working with Cloud Service projects, but that’s not a requirement.
The next step is selecting the combination of web and worker roles that you want to encapsulate within the cloud service. There’s a few flavors of the roles depending on your needs:
- ASP.NET Web Role – a default ASP.NET application, such as you’d use for Web Forms development
- ASP.NET MVC 2 Web Role – an ASP.NET project set up with the MVC (model-view-controller) paradigm, including the Models, Views, and Controllers folders and complementary IDE support
- WCF Service Web Role – a WCF project set up with a default service and service contract
- Worker Role – a basic worker role implementation (essentially a class library with a Run method in which you’ll put your code)
- CGI Web Role – a web role that runs in IIS under the Fast CGI module. Fast CGI enables the execution of native code interpreters, like PHP, to execute within IIS. If you’re looking for more information on this topic, check out Colinizer’s blog post.
For Azure@home we need only an ASP.NET web role and a worker role, which you can see I added above. Selecting OK in this dialog results in the creation of three projects: one for each of the roles and one for the Cloud Service itself. There are a few interesting things to note here.
- WebRole and WorkerRole each come with role-specific class files, providing access points to lifecycle events within the cloud. In the WebRole, you might not need to touch this file, but it’s a good place to put initialization and diagnostic code prior to the ASP.NET application’s start event firing. Beyond that though, the WebRole files look identical to what you’d get if you were building an on-premises application, and indeed a vast majority of constructs you’d use in on-premises applications translate well to Azure.
- There is a one-to-one correspondence between the role projects and an entry beneath the Roles folder within the Cloud Service project. Those entries correspond to the configuration options for each of the roles. All of the configuration is actually contained in the ServiceConfiguration.cscfg and ServiceDefinition.csdef files, but these Role nodes provide a convenient way to set properties via a property sheet versus poring over the XML. …
Jim continues with his illustrated tutorial.
Kyle McClellan reported “IntelliTrace is not supported for the RIA Services framework” in his RIA, Azure, and IntelliTrace of 6/9/2010 to which Jim Nakashima linked on 7/22/2010:
System.Security.VerificationException: Operation could destabilize the runtime
System.ServiceModel.DomainServices.Server.DomainServiceDescription.GetQueryEntityReturnType
System.ServiceModel.DomainServices.Server.DomainServiceDescription.IsValidMethodSignature
System.ServiceModel.DomainServices.Server.DomainServiceDescription.ValidateMethodSignature
System.ServiceModel.DomainServices.Server.DomainServiceDescription.AddQueryMethod
System.ServiceModel.DomainServices.Server.DomainServiceDescription.Initialize
System.ServiceModel.DomainServices.Server.DomainServiceDescription.CreateDescription
<>c__DisplayClass8.AnonymousMethod
System.ServiceModel.DomainServices.Server.DomainServiceDescription.GetDescription
System.ServiceModel.DomainServices.Hosting.DomainServiceHost..ctor
System.ServiceModel.DomainServices.Hosting.DomainServiceHostFactory.CreateServiceHost
System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost
HostingManager.CreateService
HostingManager.ActivateService
HostingManager.EnsureServiceAvailable
System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest
System.ServiceModel.AspNetPartialTrustHelpers.PartialTrustInvoke
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequestWithFlow
ScheduledOverlapped.IOCallback
IOCompletionThunk.UnhandledExceptionFrame
System.Threading._IOCompletionCallback.PerformIOCompletionCallbackOn the client, this will appear as the standard “Not Found” CommunicationException.
There are three workarounds for this problem. The first is to disable IntelliTrace altogether. This is the easiest option, but it has the broadest impact. You can disable IntelliTrace by unchecking the checkbox at the bottom of the Publish window.
The second workaround is to only collect Azure events without collecting call information. RIA Services raises some basic Azure events, so this option does have a small advantage over the final option. However, once again we’ve used a broad solution that prevents us from harnessing the full power of IntelliTrace. You can select the events only option in the IntelliTrace Settings window available on the Publish window.
The last solution is to exclude the RIA Services framework libraries from IntelliTrace call information collection. This solution enables full IntelliTrace support for the rest of your application. On the Modules tab of the IntelliTrace Settings window, you will need to add an exception for *System.ServicesModel.DomainServices.*.
I’ve tried to capture the significant points about this error so you won’t have to worry much about it yourself. The IntelliTrace bug has been reported to Visual Studio and I’m expecting a fix in their first SP. I’d assume we won’t see that for a while judging by past releases, but I’ll keep you updated if there are better fixes available later on. So for now, just pick a workaround and don’t look back.
Return to section navigation list>
Mike West, Robert McNeill and Bruce Guptil co-authored One-Stop Shopping – Major Vendors Acquire Assets for the Cloud, a Research Alert of 7/22/2010 for Saugatuck Technology (site registration required):
What is Happening? The IT industry is in a major transition, impacted by several trends concurrently that have led to increasing partnering and consolidation, as major vendors seek to become the sole source for offerings up and down the IT EcoStack™ targeting the Cloud (please see Strategic Perspective, “Gorillas In the Cloud: Applying Saugatuck’s “Master Brand” Model to Cloud IT,” MKT-732, published 5 May 2010). Dell, HP, IBM, Microsoft, Oracle, and EMC/VMware are among the many examples of IT vendors pursuing a vertically-integrated stack approach to Cloud IT (please see Note 1 for related Saugatuck research on the Cloud strategies of these vendors).
The latest is NTT Group’s proposed $3.2B cash offer to acquire Dimension Data, a global IT services and solutions provider. Together, NTT Group and Dimension Data hope to become a one-stop, sole source provider with offerings that complement NTT Group’s telecommunications technology. Geographic expansion is also in the mix. NTT Group’s business mainly focuses on Asia and covers Europe and the USA, while Dimension Data also covers Africa, the Middle East and Australia. Thus the coverage areas of the two businesses are highly complementary. …
The authors continue with the usual “Why is it Happening?” and “Market Impact” topics.
CloudTweaks posted Vertical Industry Use Of Cloud Computing – Who’s Getting Involved? on 7/23/2010:
Here are some useful cloud based chart illustrations on the future and direction of cloud computing.
Thanks to our friends at: MimeCast and Famapr for the research and insights.
Liz MacMillan claims “Cloud Computing has Alleviated Resource Pressures and Improved End-user Experience” in a preface to her Cloud Computing Delivering on its Promise but Doubts Still Hold Back Adoption post of 7/23/2010 about MimeCast’s research report:
Mimecast, a leading supplier of cloud-based email security, continuity, policy control and archiving, today announced the results of the second annual Mimecast Cloud Adoption Survey, an annual research report examining attitudes to cloud computing services amongst IT decision-makers in UK and US businesses. The survey, conducted by independent research firm, Loudhouse, reveals that a majority of organisations (51 percent) are now using some form of cloud computing service, and the levels of satisfaction amongst those companies is high across the board. Conversely, companies not yet using cloud services cite concerns around cost and security.
The survey shows that of those businesses using cloud services, 74 percent say that the cloud has alleviated internal resource pressures, and 72 percent report an improved end-user experience. 73 percent have managed to reduce their infrastructure costs, while 57 percent of say that the cloud has resulted in improved security. However, not everyone is convinced. 74 percent of IT departments still believe that there is always a trade-off between cost and IT security and 62 percent say that storing data on servers outside of the business is always a risk.
Highlights from the research:
Cloud services are now the norm:
The majority of organisations now use cloud-based services. The report found 51 percent of organisations are now using at least one cloud-based application. Adoption rates amongst US businesses are slightly ahead of the UK with 56 percent of respondents using at least one cloud-based application, compared to 50 percent in the UK. This is a significant rise from the 2009 survey, when just 36 percent of US businesses were using cloud services.
Two thirds of businesses are considering adopting cloud computing. Encouragingly for vendors, 66 percent are now considering adopting cloud-based applications in the future. Again, US businesses are ahead of the UK in their attitudes to the cloud with 70 percent considering cloud services, compared to 60 percent in the UK.
Email, security and storage are the most popular cloud services. 62 percent of the organisations that use cloud computing are using a cloud-based email application. Security and storage are the next most popular, used by 52 percent and 50 percent of organisations with at least one cloud-based service respectively. Email services are most popular with mid-size businesses (250-1,000 employees) with 70 percent of these organisations using the cloud to support this function. Smaller businesses (under 250 employees) are most likely to use the cloud for security services, with larger enterprises (over 1,000 employees) most likely to make use of cloud storage services.
Cloud attitudes split between the 'haves' and 'have-nots':
Existing cloud users are satisfied. Security is not considered to be an issue by existing cloud users: 57 percent say that moving data to the cloud has resulted in better security, with 58 percent saying it has given them better control of their data. 73 percent of current cloud users say it has reduced the cost of their IT infrastructure and 74 percent say it has alleviated the internal resource pressures upon the department.
Security fears are still a barrier. Three quarters (74 percent) of IT departments agreed with the statement 'there is always a trade-off between cost and IT security', suggesting that many organisations feel cloud solutions are less secure than the more expensive, on-premise alternatives, simply due to their lesser cost. 62 percent believe that storing data on servers outside of the business is a significant security risk.
IT is concerned that adopting cloud will not initially result in cost savings. 58 percent of respondents thought that replacing legacy IT solutions will almost always cost more than the benefits of new IT.
Cloud concerns stem from a lack of clarity. One reason for the negative perceptions of cloud services among non-users seems to be a lack of clear communication from the industry itself. 54 percent of respondents said the potential benefits of cloud delivery models are overstated by the IT industry. …
<Return to section navigation list>
No significant articles today.
<Return to section navigation list>
Lori MacVittie (@lmacvittie) asks Web 2.0 and cloud computing have naturally pushed all things toward application-centric views, why not the VPN? in a preface to her F5 Friday: Beyond the VPN to VAN post of 7/23/2010 to F5’s DevCentral blog:
When SSL VPNs were first introduced they were a welcome alternative to the traditional IPSEC VPN because they reduced the complexity involved with providing robust, secure remote access to corporate resources for externally located employees.
Early on SSL VPNs were fairly simple – allowing access to just about everything on the corporate network to authenticated users. It soon became apparent this was not acceptable for several reasons, most prominently standing out the risk of infection by remote employees who might have been using personal technology to work from home. While most organizations have no issue with any employee working a few extra hours at home, those few extra hours of productivity can be
easily offset by the need to clean up after a virus or bot entering the corporate network from an unsecured, non-validated remote source. This was especially true as one of the selling points for SSL VPN was (and still is) that it could be used from any endpoint. The “clientless” nature of SSL VPN made it possible to use a public kiosk to log-in to corporate resources via an SSL VPN without fear that the ability to do so would be “left behind.” I’m not really all that sure this option was ever widely used, but it was an option.
Then SSL VPNs got more intelligent. They were able to provide endpoint security and policies such that an “endpoint”, whether employee or corporate owned, had to meet certain criteria – including being “clean” – before it was allowed access to any corporate resource. This went hand in hand with the implementation of graded authentication, which determined access rights and authorization levels based on context: location, device, method of access, etc… That’s where we sat for a number of years. There were updates and upgrades and additions to functionality but nothing major about the solution changed.
Until recently. See, the advent of cloud computing and the increasing number of folks who would like to “work from home” if not as a matter of course then as a benefit occasionally has been driving all manner of solutions toward a more application-centric approach and a more normalized view of access to those applications. As more and more applications have become “webified” it’s made less sense over time to focus on securing remote access to the corporate network and more sense to focus on access to corporate applications – wherever they might be deployed.
THE NEXT GENERATION of ACCESS CONTROL
That change in focus has led to what should be the next step in the evolution of remote access – from SSL VPN to secure access management, to managing application access by policy across all users regardless of where they might be located.
Similarly, it shouldn’t matter whether corporate applications are “in the cloud” or “in the data center”. A consistent method of managing access to applications across all deployment locations and all users reduces the complexity inherent in managing both sides of the equation.
We might even call this a Virtual Application Network (VAN) instead of a Virtual Private Network (VPN) because what I’m suggesting is that we create a “network” of applications that is secured by a combination of transport layer security (SSL) and controlled by context-based access management at the application layer. Whether a user is on the corporate LAN or dialed-in from some remote location that has yet to see deployment of broadband access shouldn’t matter. The pre-access validation that the accessing system is “clean” is just as important today when the system is local as if it were remote; viruses and bots and malware don’t make the distinction between them, why should you?
By centralizing application access across users and locations, such secure access methodologies can be used to extend control over applications that may be deployed in a cloud computing environment as well. Part of F5’s position on cloud computing is that many of the solutions that will be required to make cloud-deployed applications viable is that the control that exists today over locally deployed applications must be extended somehow to those remote applications as a means to normalize management and security as well as controlling the costs of leveraging what is supposed to be a reduced cost environment.
That’s part of the promise of F5’s BIG-IP Access Policy Manager (APM). It’s the next step in secure remote access that combines years of SSL VPN (FirePass) experience with our inherent application-aware delivery infrastructure. It provides the means by which access to corporate applications can be normalized across users and application environments without compromising on security and control. And it’s context-aware because it’s integrated into F5’s core enabling technology platform, TMOS, upon which almost all other application delivery functionality is based and deployed.
I highly encourage a quick read of George Watkin’s latest blog on the topic, Securing the Corporate Intranet with Access Policy Manager, in which he details the solution and some good reasons behind why you’d want to do such a thing (in case I’m not convincing enough for you). You may also enjoy a dive into a solution presented in a previous F5 Friday, “F5 Friday: Never Outsource Control”, that describes an architectural approach to extending normalized control of application access to the cloud.
Chris Czarnecki asserts Security is Virtually Different in the Cloud and offers a cloud-computing white paper in this 7/23/2010 post to Learning Tree’s Cloud Computing blog:
I have just taught a version of the Learning Tree Cloud Computing course and top of the agenda was security and enough debate to stimulate this posting. Security is important in the cloud but is it really that different to security in general application and data security stored on private networks ? The answer is yes most probably.
Security of data and application security principles applied to private networks and deployments should still be applied to the cloud of course. Doug Rehnstrom posted on this recently. Security in the cloud is probably different from a private network and one of the major reasons is virtualization.
Cloud technology is built upon virtualization – this raises a number of security concerns – not just for the cloud but for all organisations that use virtualisation technology. The security of a virtualised solution is highly dependent on the security of each of its independent components – this has been highlighted recently by NIST who have issued guidelines on security in virtualised environments.
Security in a virtualised environment depends on the security of the hypervisor, the host operating system, guest operating system, applications, storage devices, networks connecting them. How many organisations that have deployed virtualised environments – and thats a lot, have actually considered the security implications of their implementation. I am confident that many of these organisations are the ones who state security as a barrier to adopting the cloud. As private clouds become more prevalent then the security of the virtualization, its monitoring and compromise detection will need to be carefully considered and adopted. Should that not be the case for all virtualized deployments, cloud or not ? Most definitely yes too. So if you are using a virtualized environment your security requirements are not so different from the cloud, you just may not have realised it.
If you are interested in the discussion further have a look at the white paper I recently put together. [Requires site registration.]
<Return to section navigation list>
Blackhat announced Blackhat USA Uplink 2010 on 7/23/2010. For US$395, you receive:
Streaming live from Black Hat USA • July 28-29
This year thousands of security professionals from around the world are making plans to be a part of Black Hat USA 2010. But not all of those people will actually be in Las Vegas. With Black Hat Uplink, you can experience essential content that shapes the security industry for the coming year - for only $395.
Now for the first time, you can get a taste of Black Hat USA from your desk - this year's live event will be streamed directly to your own machine with Black Hat Uplink:
- Access to two select tracks on each day of the Briefings and the keynote - a total of 20+ possible sessions to view.
- Post-conference access to Uplink content; go back and review the presentations that you missed or watch the presentations that interested you the most as many times as you want.
- Interact with fellow con-goers, Uplink attendees, and the security community at large via Twitter during the Briefings.
- Get show promotional pricing for the “Source of Knowledge” DVDs should you wish to purchase ALL of the recordings from Black Hat USA and/or DEF CON 18.
Presentations will be streamed live on July 28-29, but you will be able to view Uplink presentations for up to 90 days after the event.
Gartner listed cloud-focused events for 2010 in a 7/23/2010 e-mail message:
Gartner Outsourcing & Vendor Management Summit
September 14 – 16 I Orlando, FL
Learn how cloud computing fits into an effective sourcing strategy, what terms and conditions, delivery and pricing to expect, and how to select, manage and govern cloud services and service providers. View details on our Global Sourcing Invitational Program.
Dedicated cloud trackGartner Symposium/ITxpo
October 17 – 21 I Orlando, FL
Learn to build or validate your cloud strategy, measure cloud investments, evaluate cloud and software-as-a-service platforms and providers, and manage security and risk related to both building a cloud-enabled IT organization and taking advantage of cloud business opportunities.
Dedicated track on cloudGartner Application Architecture, Development & Integration Summit
November 15 – 17 I Los Angeles, CA
Assess the latest cloud trends, adoption best practices, key benefits and risks, and get a grip on the tools and techniques to architect, develop and integrate applications and services in the cloud. Stay tuned for details on a dedicated cloud track.Gartner Data Center Conference
December 6 – 9 I Las Vegas, NV
Understand the implications of cloud computing across the data center and the infrastructure and operations organization, and get an in-depth perspective on private clouds and other cloud-enabled infrastructure issues. Stay tuned for details on a dedicated cloud track.Upcoming Gartner webinar focused on cloud:
Server Virtualization: From Virtual Machines to Clouds
Thomas J. Bittman, Vice President and Distinguished Analyst
Wednesday, July 28, 2010, 9 a.m. – 10 a.m. EDT
Wednesday, July 28, 2010, 12 p.m. – 1 p.m. EDTView 2010 Gartner events: See our complete listing of 2010 Gartner events.
Stephen Forte’s Devopalooza at VS Live! Redmond post of 7/23/2010 announced his presentations at VS Live! Redmond:
In just over a week, I will be speaking at VS Live! at Microsoft’s campus in Redmond, Washington. In addition to my Scrum and OData talks, I will be helping host Devopalooza on Wednesday August 4th at 6:30-8:30pm in Building 33. We have a lot of cool stuff ready for the attendees, including a “Developer Jeopardy” contest. Brian Peek and I will be the hosts of the game show and we have been working hard coming up with good questions.
Yes, there will be prizes. Yes there will be lots of wise cracks. And yes I will be wearing my rugby jersey.
<Return to section navigation list>
Alex Williams asserts that Opsource causes The Cloud Services Market [to Look] Like a Red Ocean in this 7/23/2010 post to the ReadWriteCloud blog:
Earlier this week, Opsource announced a partner program. The news came on Monday, the first day developers could download code from OpenStack, a separate initiative that has had considerable attention this week.
Both companies provide cloud infrastructure based upon open-source. They could easily be part of a same open cloud network such as OpenStack. But for now Opsource says they will not join the effort lead by Rackspace. Our guess is Opsource competes to some extent with Rackspace and is looking at other alternatives.
Rackspace, Opsource and almost two dozen other companies are now offering some variety of cloud infrastructure. It's an increasingly crowded market that will be due for some consolidation. But for the moment, the market is a picture of how companies perceive the opportunities the cloud provides
Rackspace, as we know, provides public cloud infrastructure. Opsource offers cloud infrastructure to the enterprise, service providers and systems integrators.
Opsource calls its program a partner ecosystem. These partners include integrators, developers, ISVs, cloud platform companies and telecom providers.
Opsource expects 50% of its revenue to come from these partners. Opsource points in particular to telecommunications companies. CEO Treb Ryan:
"The feedback we're getting from the telecoms is that a request for cloud is becoming an increasing part of the RFP's they are seeing from customers. Usually it comes as a request for bundled services (such as managed network, internet access, hosting and cloud.) One of our European telecoms stated they couldn't bid on $2 million a month worth of contracts because one of the requirements was cloud."
Ryan says customers want to work with one company:
"Most likely it's a combination of not wanting to go to separate vendors for separate services (i.e. a colo company for hosting, OpSource for cloud, a telecom for network, and a managed security company for VPN's) they want to get it all from one vendor completely integrated. Secondly, I think many customers have a trusted relationship with their telecom for IT infrastructure services already and they trust them more than a third-party company."
Opsource and Rackspace are two well-established companies in the cloud computing space. But the number of providers in the overall market is beginning to morph.
John Treadway of Cloud Bzz put together a comprehensive list. He says the market is looking more like a red ocean:
"I hope you'll pardon my dubious take, but I can't possibly understand how most of these will survive. Sure, some will because they are big and others because they are great leaps forward in technology (though I see only a bit of that now). There are three primary markets for stacks: enterprise private clouds, provider public clouds, and public sector clouds. In five years there will probably be at most 5 or 6 companies that matter in the cloud IaaS stack space, and the rest will have gone away or taken different routes to survive and (hopefully) thrive."
Lots of blood in the water. Who's going to get eaten first?
Maureen O’Gara prefaced her Unisys to Offer Price Fixe Cloud post of 7/23/2010 with “The environment can be used to integrate SOA technology:”
In what is believed to be a cloud first, Unisys is going to offer its upscale enterprise-class ClearPath users a fixed-priced PaaS cloud, a model that flies in the face of the nickel and dime'ing that goes on in commodity cloud land.
Unisys already has x86-based commodity clouds on offer but now it's drawn its proprietary mainframe-style ClearPath widgetry, based on its MCP and OS/2000 operating systems, into the new meme beginning with a managed development and testing solution that will go for $13,000 for three months use of a soup-to-nuts environment that includes 25MIPS, eight megs [gigs?] of memory and 75GB of storage. [Emphasis added.]
Unisys has a couple of other fancier configurations but it thinks this so-called basic configuration will suit 90% of its markets needs. And, to repeat, it includes everything: CPUs, memory, storage, networking, bandwidth, operating system, SDKs, maintenance, support and enhancements.
If a customer believes it needs more than three months use of the widgetry it has to buy another three-month block.
The company is starting with test & dev and means to branch out into production environments, managed SaaS solutions and data replication and disaster recovery. The production environment might suit smaller or specialized apps and large accounts that need an overflow environment.
The widgetry will be hosted in Unisys' five data centers where, it says, the infrastructure will be "over-provisioned."
Unisys has been using the test & dev stuff internally for the last couple of years to save capex costs but tidied-up the interfaces when it decided to commercialize it. That's what's taken it a while to come to market.
It says the environment can be used to develop or enhance native ClearPath apps, integrate SOA technology to create composite apps or add functions like business intelligence. The development environment can accommodate Unisys Business Information Server (BIS) and its Enterprise Application Environment (EAB)/AB Suite among other development environments.
The company already has some SaaS-style solutions running on ClearPath like its Logistics Management System where 35% of the world's airline cargo is tracked and traced.
Outside of Japan Unisys has 1,600 ClearPath customers and estimates that 5%-10% will immediately be interested in the ClearPath Cloud, which of course keeps the stuff relevant. In a recent survey Unisys found the test & dev cloud was attractive to 46% of the respondents.
That’s US$52,000 per year, folks, but I’m sure you get more than “eight megs of memory” with that.
Pat Romanski adds fuel to the ClearPath fire by claiming “Unisys-hosted solutions give ClearPath clients new, cost-efficient options” in a preface to her New Unisys ClearPath Solutions Propel Clients into the Cloud post of 7/23/2010:
ClearPath Cloud Development and Test Solution is a "platform as a service" (PaaS) solution. It extends Unisys strategy for delivering secure IT services in the cloud and sets the stage for delivery of further ClearPath Cloud solutions in the near future.
"With the ClearPath Cloud Solutions, Unisys enables ClearPath clients to leverage a new, cost-efficient business model for use of IT resources," said Bill Maclean, vice president, ClearPath portfolio management, Unisys. "Clients can subscribe to and access Unisys-owned computing facilities incrementally to modernize existing ClearPath applications or develop new ones. They can avoid unscheduled capital equipment expenditures in this uncertain economy and make more efficient use of their existing ClearPath systems for applications critical to their business mission.
"Establishing this new cloud-based server environment for delivery of ClearPath development services is just the latest step Unisys has taken to give clients new ways to capitalize on what we believe is the most modern, secure, open mainframe platform on the market," Maclean added.
New Test and Development Solution Makes It Easy to Jump into the Cloud
In a recent Unisys online poll, 46 percent of respondents - the largest single percentage response to any question in the poll - cited development and test environments as the first workloads they would consider moving to the cloud. The Unisys ClearPath Cloud Development and Test Solution, hosted at Unisys outsourcing center in Eagan, Minnesota and available to clients around the world, simplifies the transition to the cloud.The solution is designed to supplement clients' current development and test environments, providing access to additional virtual resources when needed for creation, modernization and functional testing of specific applications. It provides expanded infrastructure sourcing options; accelerates delivery of business functionality to help clients hold down costs and keep projects on schedule; and provides a new approach that helps minimize the risks of large, costly IT projects.
Clients who choose the ClearPath Cloud Development and Test Solution purchase a three-month development-and test-block license to use virtual resources in the Unisys hosted server cloud environment. This gives them access to a ready-to-go development environment that includes:
- A Software Developer's Kit (SDK) for the Unisys MCP or OS 2200 operating environment, including relevant database software and development tools;
- A virtual development server, with amounts of memory, storage and networking resources sized to the specific development and testing requirement; and
- Software enhancement releases and maintenance support services.
Clients can use this solution to develop or enhance native ClearPath applications, integrate SOA technology to create composite applications or add functions such as business intelligence. The development environment can accommodate Unisys Business Information Server (BIS), Enterprise Application Environment (EAE)/AB Suite and other development environments.
Unisys engineering organization has deployed this solution for its internal system software development and testing, cutting virtual server provisioning time from days to less than an hour while avoiding the cost of additional capital equipment.
Services Amplify Value of ClearPath Cloud Development and Test Solution
Unisys offers optional Development and Test Planning and Setup Services to help clients define, implement and tune the best development environment for their specific initiative.Clients can also purchase Unisys Cloud Transformation Services, which help the client design and implement the right cloud strategy to support their evolving business needs. Plus, those using the solution for application modernization can opt for Unisys Application Advisory Solutions. These services help the client determine how best to streamline the environment for maximum return on investment - for example, consolidating applications, integrating SOA capabilities, or enabling wireless, web, or social-media access.
The ClearPath Cloud Development and Test Solution draws on the infrastructure behind Unisys Secure Cloud Solution for provisioning and resource management - including Unisys Converged Remote Infrastructure Management Suite solution offering, which provides a unified view of an entire IT infrastructure to facilitate integrated management.
Future ClearPath Cloud Solutions for Enhanced Production and Recovery
Over the next several months, Unisys plans to launch additional ClearPath Cloud Solutions that, when available, will give clients new options for realizing the operational and economic advantages of cloud services. The currently planned solutions include:
- A full cloud-based production environment for smaller, independent applications. This is intended to enable clients to offload specialized applications into the cloud and maximize use of their data-center resources for mission-critical applications;
- Data replication and disaster recovery for cost-effective, secure preservation of mission-critical business information; and
- Managed, "software as a service" application solutions for clients' specific industry requirements. These are intended to complement Unisys existing SaaS solutions, such as the Unisys Logistics Management System for air cargo management.
US$52,000 per year still sounds to me like a lot of money for what’s described above.
Redmonk’s Michael Coté posted IBM’s New zEnterprise – Quick Analysis about another mainframe entry in the cloud-computing sweepstakes on 7/22/2010:
IBM launched a new mainframe this morning here in Manhattan, the zEnteprise – a trio of towers bundling together mainframe (the new 196) along with stacks of Power/AIX and x86 platforms into one “cloud in a box,” as Steve Mills put it. As you can guess, the pitch from IBM was that this was a more cost effective way of doing computing than running a bunch of x86 commodity hardware.
(To be fair, there was actually little talk of cloud, perhaps 1-3 times in the general sessions. This wasn’t really a cloud announcement, per se. ”Data-center in a box” was also used.) [Emphasis added.]
TCO with zEnterprise
The belief here is two-fold:
- Consolidating to “less parts” on the zEnterprise is more cost effective – there’s less to manage.
- As part of that, you get three platforms optimized for different types of computing in one “box” instead of using the one-size-fits-all approach of Intel-based commodity hardware.
Appropriately, this launch was in Manhattan, where many of IBM’s mainframe customers operate: in finance. The phrases “work-load” and “batch-management” evokes cases of banks, insurance companies (Swiss Re was an on-site customer), and other large businesses who need to process over all sorts of data each night (or in whatever period, in batch), dotting all the i’s and crossing all the t’s in their daily data.
IBM presented much information on Total Cost of Ownership (TCO) for the new zEnterprise vs. x86 systems (or “distributed” for you non-mainframe types out there). The comparisons went over costs for servers, software, labor, network, and storage. Predictably, the zEnterprise came out on-top in IBM’s analysis, costing just over half as much in some scenarios.
Time and time again, having less parts was a source for lower costs. Also, more favorable costs for software based on per-core pricing came up. In several scenarios, hardware ended up being more expensive with zEnterprise, but software and labor were low enough to make the overall comparison cheaper.
Fit to purpose
Another background trend that comes into play with the zEnterprise platform is specialized hardware, for example in the areas of encryption, data analytics, and I/O intensive applications. The idea is that commodity hardware is, by definition, generalized for any type of application and, thus, misses the boat on optimizing per type of workload. If you’re really into this topic, look for the RedMonkTV interview I did with IBM’s Guruaj Rao, posted in the near future.
While mainframes might be an optimal fit for certain applications, Rao also mentions in tbe interview, there are many innovations coming out of the distributed (x86) space. Rather than pass up on them because they don’t run on the mainframe, part of the hope of zEnterprise is to provide a compatible platform that’s, well, more mainframe-y to run distributed systems on. What this means is that you’d still run on Unix or Linux as a platform, but those servers would be housed on the zEnterprise-hosted blades; they’d be able to take advantage of the controlled and managed resourced, and be managed as one system.
On the distributed note, a couple analysts asked about Windows during the Q&A with Software and Systems Group head Steve Mills: would zEnterprise run Windows (Server. The short answer is: no. The longer one is about lack of visibility into source code, not wanting to support an OS that “drag[s] in primitives from DOS,” and generally not being able to shape Windows to the management IBM would want. Mills said he “doesn’t really every expect to manage Windows” on zEnterprise.
During that same Q&A, Mills also alluded to one of the perfect heterogeneity in one box scenarios that the zEnterprise seems a good technological fit for: many mainframe-based applications are served by a client and middle-ware tiers for their user-facing layers, 3-tier applications, if you will, with the mainframe acting as the final tier, the system of record, to use the lingo. You could see where the web/UI tier ran on x86, the middleware & integration on AIX/Power, and the backend on z.
The Single System, for vendors, for buyers
“Everything around the server has become more expensive than the server itself.”
–Steve Mills, IBM
The word “unified” comes up in the zEnterprise context frequently. Indeed, the management software for zEnterprise is called “Unified Resource Manager.” As with other vendors – Cisco, Oracle, and private cloud platforms to some extent – the core idea here is having one, system that acts like a homogenous platform…all the while being heterogenous underneath, smoothing it over with interfaces and such.
For vendors, the promise is capturing more of the IT budget, not just the components they specialize in (servers, database, middleware, networking, storage, applications, etc.). There’s also the chance to compete on more than just pricing, which is a nasty way to go about making money from IT.
For buyers, the idea is that by having a single-sourced system, there’s more control and integration on the box, and that increased control leads to more optimization in the form of speed, breadth of functionality, and cost savings. There’s also reduced data center foot-print, power consumption, and other TCO wing-dings.
Sorting out the competing scenarios of heterogeneous vs. unified systems isn’t a simple back of the napkin affair: there’s so much apples to oranges comparisons that it’s tough to balance anything but the final bill. Part of the success of non-mainframe infrastructure is that its dumb-simplicity makes evaluating it clear and straight forward: x86-based servers have standardized computer procurement. Layering in networking, software, storage, management, and all that quickly muddies it back up, but initially it’s a lot more clear-cut than evaluating a new type of system. And it certainly feels good to buy from more than one vendor rather than put all your eggs in one gold-plated basket.
Nonetheless, you can expect vendors to increasingly look to sell you a single, unified system. To evaluate these platforms, you need a good sense of the types of applications you’ll be running on them: what you’ll be using them for. There’s still a sense that a non-mainframe system will be more flexible, agile even…but only in the short term, after which all that flexibility has created a mess of systems to sort out. Expect much epistemological debate over that mess.
More
Patrick Thibodeau at Computerworld covers many of the details: “the zEnterprise 196…includes a 5.2-GHz quad processor and up to 3TB of memory. That’s double the memory of the preceding system, the z10, which had a 4.4-GHz quad processor.”
Timothy Prickett Morgan gets more detailed on the tech-specs and possible use-patterns: “there is no way, given the security paranoia of mainframe shops, that the network that interfaces the mainframe engines and their associated Power and x64 blades to the outside world will be used to allow Power and x64 blades to talk back to the mainframes.”
Richi Jennings [no relation] wraps up lots of coverage.
Disclosure: IBM, Microsoft, and other interested parties are clients. IBM paid for my hotel and some meat from a carving station that I ate last night.
David Worthington covered the same IBM press conference in his A new onramp for the mainframe superhighway article of 7/23/2010 for SDTimes.
<Return to section navigation list>
by --rj (roger_jennings@compuserve.com) at July 26, 2010 08:50 AM
This year looks to be another swell get-together in Vegas. I had to miss last year (first time in…forever) so I’m looking forward to 112 degrees, recirculated air, and stumble-drunk hax0rs jackpotting ATMs and commandeering elevators.
I’ll be getting in on the 27th. I have a keynote at the Cloud Security Alliance Summit on the 28th (co-located within Black Hat,) a talk on the 29th at Black Hat (Cloudinomicon) from 10am-11am and I’ll be on another FAIL panel at Defcon with the boys. I’ve got a bunch of (gasp!) customer meetings and (gasp! x2) work stuff to do, but plenty of time for the usual.
I’m going to try to hit Cobra Kai, Xtreme Couture or the Tapout facilities whilst there for some no-gi grappling or even BJJ if I can find a class. Either way, there are some hard core P90X’ers that I’m sure I can con into working out in 90 degree, 6am weather.
Rumors of mojitos and cigars at Casa Fuente are completely unfounded. Completely.
Oh, parties? They have parties? 
See y’all there!
/Hoff
I wasn’t at the OSCON Cloud Summit this past week, but I’ve spent some time over the weekend trying to collect the good bits. Via Twitter, I had heard echos of an interesting debate on Cloud standards between Sam Johnston and Benjamin Black. Today I got to see Benjamin’s slides and read reports from two audience members, Charles Engelke and Krishnan Subramanian. Sam argued that Cloud standards are needed, Benjamin that they would be premature.
Benjamin is right about what to think and Sam is right about what to do.
Let me put it differently: Benjamin is right in theory, but it doesn’t matter. Here is why.
Say I’m a vendor and Benjamin convinces me
Assume I truly believe the industry would be better served if we all waited. Does this mean I’ll stay away from Cloud standards efforts for now? Not necessarily, because nothing is stopping my competitors from doing it. In the IT standards world, your only choice is to participate or opt out. For the most part you can’t put your muscle towards stopping an effort. Case in point, Amazon has so far chosen to opt out; has that stopped VMWare and others from going to DMTF and elsewhere to ratify specifications as standards? Of course not. To the contrary, it has made the option even more attractive because when the leader stays home it is a lot easier for less popular candidates to win the prize. So as a vendor-who-was-convinced-by-Benjamin I now have the choice between letting my competitor get his specification rubberstamped (and then hit me with the competitive advantage of being “standard compliant” and even “the standard leader”) or getting involved in an effort that I know to be counterproductive for the industry. Guess what most will choose?
Even the initial sinner (who sets the wheels of premature standardization in motion) may himself be convinced that it’s too early for Cloud standards. But he has to assume that one of his competitors will make the move, and in that context why give them first mover advantage (and the choice of the battlefield). It’s the typical Tragedy of the Commons scenario. By acting in a rational and self-interested way, participants invariably end up creating a bad situation, one that they might all know is against everyone’s self interest.
And it’s not just vendors.
Say I’m an officer of a Standard-setting organization and Benjamin convinces me
If you expect that I would use my position in the organization to prevent companies from starting a Cloud standard effort there, you live in fantasy-land. Standard-setting organizations compete with one another just as fiercely as companies do. If I have achieved a position of leadership in a given standard organization, the last thing I want is to see another organization lay claims to a strategic and fast-growing area of the IT landscape. It takes a lot of time and money for a company to get elected on the right board and gets its employees (or other reliable allies) in the right leadership positions. Or to acquire people already in that place. You only get a return on that investment if the organization manages to be the one where the key standards get created. That’s what’s behind the landgrab reflex of many standards organizations.
And it goes beyond vendors and standards organizations
Say I’m an IT buyer and Benjamin convinces me
Assume I really believe Cloud standards are premature. Assume they get created anyway and I have to choose between a vendor who supports them and one who doesn’t. Do I, as a matter of principle, refuse consider the “standard-compliant” label in my purchasing decision? Even if I know that the standard shouldn’t have been created, I also know that, all other things being equal, the “standard-compliant” product will attract more tools and complementary solutions and will likely ease future integration problems.
And then there is the question of how I’ll explain this to my boss. Will Benjamin be by my side with his beautiful slides when I am called in an emergency meeting to explain to the CIO why we, unlike the competitors, didn’t pick “a standards-based solution”?
In the real world, the only way to solve problems caused by the Tragedy of the Commons is to have some overarching authority regulate the usage of the resource at risk of being ruined. This seems unlikely to be a workable solution when the resource is not a river to protect from sewer discharges but an IT domain to protect from premature standardization. If called, I’d be happy to serve as benevolent dictator for the IT industry (I could fix a few other things beyond the Cloud standards landgrab issue). But as long as neither I nor anyone else is in a dictatorial position, Benjamin’s excellent exposé has no audience for which his call to arms (or rather to lay down the arms) is actionable. I am not saying that everyone agrees with Benjamin, but that even if everyone did it still wouldn’t make a difference. Many of us in the industry share his views and rationally act as if we didn’t.
[UPDATED 2010/7/25: In a nice example of Blog/Twitter synergy, minutes after posting this I was having a conversation on Twitter with Benjamin Black about my interpretation of what he said. Based on this conversation, I realize that I should clarify that what I mean by "standards" in this post is "something that comes out of a standard-setting organization" (whether or not it gets adopted), in other words what Benjamin calls a "standard specification". He uses the word "standard" to mean "what most people use", which may or may not be a "standard specification". That's a big part of the disconnect that led to our Twitter chat. The other part is that what I presented as Benjamin's thesis in my post is actually only one of the propositions in his talk, and not even the main one. It's the proposition that it is damaging for the industry when a standard specification comes out of a standard organization too early. I wasn't at the conference where Benjamin presented but it's hard to understand anything else out of slide 61 ("standardize too soon, and you lock to the wrong thing") and 87 ("to discover the right standards, we must eschew standards"). So if I misrepresented him I believe it was in making it look like this was the focus of his talk while in fact it was only one of the points he made. As he himself clarified for me: "My _actual_ argument is that it doesn't matter what we think about cloud standards, if they are needed, they will emerge" (again, in this sentence he uses "standards" to mean "something that people have converged on").
More generally, my main point here has nothing to do with Benjamin, Sam and their OSCON debate, other than the fact that reading about it prompted me to type this blog entry. It's simply that there is a perversion in the IT standards landscape that makes it impossible for premature standardization *not* to happen. It's something I've written before, e.g. in this post:
Saying “it’s too early” in the standards world is the same as saying nothing. It puts you out of the game and has no other effect. Amazon, the clear leader in the space, has taken just this position. How has this been understood? Simply as “well I guess we’ll do it without them”. It’s sad, but all it takes is one significant (but not necessarily leader) company trying to capitalize on some market influence to force the standards train to leave the station. And it’s a hard decision for others to not engage the pursuit at that point. In the same way that it only takes one bellicose country among pacifists to start a war.
Benjamin is just a messenger; and I wasn't trying to shoot him.]
Related posts:
by William (@vambenepe on Twitter) at July 25, 2010 10:07 PM
A few weeks ago I saw some RT’s/@’s on Twitter referencing John Flowers and that name brought back some memories.
Today I sent a tweet to John asking him if I remembered correctly that he was at SANS in New Orleans in 1999 when he was still at Hiverworld.
He responded back confirming he was, indeed, at SANS ’99. I remarked that this was where I first met many of today’s big names in security: Ed Skoudis, Ron Gula, Marty Roesch, Stephen Northcutt, Chris Klaus, JD Glaser, Greg Hoglund, and Bruce Schneier.
John responded back:
I couldn’t agree more. That was an absolutely amazing time. I was on my second security startup (NodeWarrior Networks,) times were booming and this generation of the security industry as we know it was being given birth to.
I remember many awesome things from that week:
I came back from that week knowing with every molecule in my body that even though I’d been “doing” security for 5 years already, it was exactly what I wanted to for the rest of my life.
I have Stephen Northcut to thank for that. I haven’t been to a SANS since 1999 (don’t ask me why) but I am so excited about going back in August in DC (SANS What Works In Virtualization and Cloud Computing Summit) and giving a keynote at the event.
It’s been a long time. Too long.
/Hoff
| A compendium of Windows Azure, Windows Azure Platform Appliance, SQL Azure Database, AppFabric and other cloud-computing articles. |
• Updated 7/25/2010 by adding:
Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:
To use the above links, first click the post’s title to display the single article you want to navigate.
Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)
Read the detailed TOC here (PDF) and download the sample code here.
Discuss the book on its WROX P2P Forum.
See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.
Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.
You can now freely download by FTP and save the following two online-only PDF chapters of Cloud Computing with the Windows Azure Platform, which have been updated for SQL Azure’s January 4, 2010 commercial release:
HTTP downloads of the two chapters are available for download at no charge from the book's Code Download page.
Steven Nagy deep-dives into Azure Table Storage in his Table Storage Retrieval Patterns post of 7/24/2010 to his Above the Cloud blog:
Overview Of Tables
There are three kinds of storage in Windows Azure: Tables, Blobs, and Queues. Blobs are binary large objects and Queues are robust enterprise level communication queues. Tables are non-relational entity storage mechanisms. All storage is three times redundant and available via REST using the ATOM format.
Tables can store multiple entities with different kinds of shapes. That is to say, you can safely store 2 objects in a table that look completely different. For example, a Product entity might have a name and a category, whereas a User entity might have name, date of birth, and login name properties. Despite the difference between these two objects, they can both be stored in the same table in Windows Azure Table Storage.
Some of the reasons we prefer to use Table storage over other database mechanisms (such as Sql Azure) is that it is optimised for performance and scalability. It achieves this through an innate partitioning mechanism based on an extra property assigned to the object, called ‘Partition Key’. Five objects each with different partition keys that are otherwise identical, will be stored on five different storage nodes.
There are a number of ways we can get and put entities into our table storage, and this article will address a few. However before we investigate some scenarios, we need to setup our table and the entities that will go into it.
Setting Up The Table
Before we view the ways we can interact with entities in a table, we must first setup the table. We are going to create a table called ‘Products’ for the purposes of this article. Here is some code that demonstrates how (this could go in Session or Application start events or anywhere you see fit).
var account = CloudStorageAccount.FromConfigurationSetting(“ProductStorage”);
var tableClient = account.CreateCloudTableClient();
tableClient.CreateTableIfNotExist(“Products”);The above code assumes you have a connection string configured already called ‘ProductStorage’ which points to your Windows Azure Storage account (local development storage works just as well for testing purposes).
Setting Up The Entity
For the purposes of this article we are going to put an entity called ‘Product’ into the table. That entity can be a simple POCO (plain old CLR object); only the publicly accessible properties will be persisted and retrievable however. Lets define a simple product entity with a name, category and a price. However all objects stored in tables must also have a row key, partition key, and a time stamp, otherwise we will get errors when we try to persist the item. Here’s our product class:
public class Product { // Required public DateTime Timestamp { get; set; } public string PartitionKey { get; set; } public string RowKey { get; set; } // Optional public string Name { get; set; } public string Category { get; set; } public double Price { get; set; } }Pretty simple eh? However we can clean up some of the code; because the Timestamp, PartitionKey, and RowKey are all required for every single table entity, we could pull those properties out into a base entity class. However we don’t have to; there already exists one in the StorageClient namespace called ‘TableServiceEntity’. It has the following definition:
[CLSCompliant(false)] [DataServiceKey(new string[] {"PartitionKey", "RowKey"})] public abstract class TableServiceEntity { protected TableServiceEntity(string partitionKey, string rowKey); protected TableServiceEntity(); public DateTime Timestamp { get; set; } public virtual string PartitionKey { get; set; } public virtual string RowKey { get; set; } }It makes sense for us to inherit from this class instead. We’ll also follow the convention of having a partition key and row key injected in the constructor on our Product class, while also leaving a parameterless constructor for serialisation reasons:
public class Product : TableServiceEntity { public Product() { } public Product(string partitionKey, string rowKey) : base(partitionKey, rowKey) {} public string Name { get; set; } public string Category { get; set; } public double Price { get; set; } }Done. We’ll use this Product entity from now on. All scenarios below will use a test Product with the following information:
var testProduct = new Product("PK", "1") { Name = "Idiots Guide to Azure", Category = "Book", Price = 24.99 };Scenario 1: Weakly Typed Table Service Context
The easiest way to get started with basic CRUD methods for our entity is by using a specialised ‘Data Service Context’. The Data Service Context is a special class belonging to the WCF Data Services client namespace (System.Data.Services.Client) and relates to a specific technology for exposing and consuming entities in a RESTful fashion. Read more about WCF Data Services here.
In a nutshell, a Data Service Context lets us consume a REST based entity (or list of entities) and that logic is given to us for free in the ‘DataServiceContext’ class, which can be found in the afore mentioned System.Data.Services.Client namespace (you’ll probably need to add a reference). Consuming RESTful services is not an Azure specific thing, which is why we need to import this new namespace.
Because table storage entities act exactly like other RESTful services, we can use a data services context to interact with our entity. Tables and their entities have a few additional bits surrounding them (such as credential information like the 256bit key needed to access the table storage) so we need to be able to include this information with our data context. The Azure SDK makes this easy by providing a class derived from DataServiceContext called ‘TableServiceContext’. You’ll notice that to instantiate one of these we need to pass it a base address (our storage account) and some credentials.
If you review some of the original code above, you’ll notice we created a CloudTableClient based on connection string information in our configuration file. That same table client instance has the ability to create our TableServiceContext, using the code below:
var context = tableClient.GetDataServiceContext();
That’s it! All the explanation above just for one line of code eh? Well hopefully you understand what’s happening when we get that context. It is generating a TableServiceContext which inherits from DataServiceContext which contains all the smarts for communicating to our storage table. Simple.
Now we can call all sorts of methods to create/delete/update our products. We’ll use the ‘testProduct’ defined earlier:
context.AddObject("Products", testProduct); context.SaveChanges(); testProduct.Price = 21.99; context.UpdateObject(testProduct); context.SaveChanges(); var query = context.CreateQuery<Product>("Products"); query.AddQueryOption("Rowkey", "1"); var result = query.Execute().FirstOrDefault(); context.DeleteObject(result); context.SaveChanges();The methods being called here only know about ‘object’, not ‘Product’ and are therefore not type safe. We’ll look at a more type safe example in the next scenario.
Scenario 2: Strongly Typed Table Service Context
In the previous example we saw that the Table Service Context was a generic way to get going with table entities quickly. This works well because we can put any type of entity into the table via the same ‘AddObject’ method. However sometimes in code we like to be more type safe than that and want to enforce that a particular table only accepts certain objects. Or perhaps we want unique data access classes for our different entity types so that we can put some validation in.
Either way, this is relatively easy to achieve by creating our own Data Service Context class. We still need to wrap up table storage credentials, so its actually easier if we inherit from TableServiceContext, as follows:
public class ProductDataContext : TableServiceContext { public ProductDataContext(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials) { } // TODO }The base constructor of TableServiceContext requires us to supply a base address and credentials, so we simply pass on this requirement. Our constructor doesn’t need to do anything else though.
The next step is to start adding methods to this new class that perform the CRUD operations we require. Let’s start with a simple query:
public IQueryable<Product> Products { get { return CreateQuery<Product>("Products"); } }This will give us a ‘Products’ property on our ProductDataContext that will allow us to query against the product set using LINQ. We’ll see an example of that in a minute. For now, we’ll add in some strongly typed wrappers for the other CRUD behaviours:
public void Add(Product product) { AddObject("Products", product); } public void Delete(Product product) { DeleteObject(product); } public void Update(Product product) { UpdateObject(product); }Nothing very special there, but at least we can enforce a particular type now. Let’s see how this might work in code to make calls to our new data context. As before we’ll assume the table client has already been created from configuration (see ‘Setting Up The Table’ above) and we’ll use the same test product as before:
var context = new ProductDataContext( tableClient.BaseUri.ToString(), tableClient.Credentials ); context.Add(testProduct); context.SaveChanges(); testProduct.Price = 21.99; context.Update(testProduct); context.SaveChanges(); var result = context.Products .Where(x => x.RowKey == "1") .FirstOrDefault(); context.Delete(result); context.SaveChanges();You can see the key differences from the weakly typed scenario mentioned earlier. We now use the new ProductDataContext, however we can’t automatically create it like we can with the generic table context, so we need to instantiate it ourselves, passing the base URI and credentials from the table client. We also use our more explicitly typed methods for our CRUD operations, however you might notice there is a big change in the way we query data. The ‘Products’ property returns IQueryable<Product> which means we can use LINQ to query the table store. Careful though, not all operations are supported by the LINQ provider. For example this will fail:
var result = context.Products.FirstOrDefault(x => x.RowKey == "1");.. because FirstOrDefault is not supported with predicates. However this new query API is much nicer and allows us to do a lot more than we could when the base entity type was unknown by the data context.
Scenario 3: Using The Repository and Specification Patterns
Before reading on you might want to familiarise yourself with the concepts of these patterns. To prevent blog duplication, please refer to this article that someone smarter than me wrote:
Implementing Repository and Specification patterns using Linq.
The goal is to create a repository class that can take a generic type parameter which is an entity we want to work with. Such a repository class will be reusable for all types of entities but still be strongly typed. We also want to have it abstracted via an interface so that we are never concerned with the concrete implementation. For more information on why this is good practise, please refer to the SOLID principles.
We also want to use the specification pattern to provide filter/search information to our repository. We want to leverage the goodness of LINQ but also explicitly define those filters as specifications so that they are easily identifiable.
I usually find it easiest to start with the interface and worry about the implementation later. Let’s define an interface for a repository that will take any kind of table entity:
public interface IRepository<T> where T : TableServiceEntity { void Add(T item); void Delete(T item); void Update(T item); IEnumerable<T> Find(params Specification<T>[] specifications); void SubmitChanges(); }Seems simple enough, however you might note that our find process is less flexible than in scenario 2 where we could just use LINQ directly against our data service. We want to provide the flexibility of LINQ yet still provide explicitness and reusability of those very same queries. We could add a bunch of methods for each query we want to do. For example, to retrieve a single product, we could create an extra method called ‘GetSingle(string rowkey)’. However that only applies to products, and may not apply to other entity types. Likewise, if we want to get all Products over $15, we can’t do that in our repository because it makes no sense to get all User entities that are over $15.
That’s where the specification pattern comes in. A specification is a piece of information about how to refine our search. Think of it as a search object, except it contains a LINQ expression. We’ll see with an example soon, but lets just define our specification class and adjust our Find method on our IRepository<T> interface first:
IEnumerable<T> Find(params Specification<T>[] specifications); ... public abstract class Specification<T> { public abstract Expression<Func<T, bool>> Predicate { get; } }Our Find method has been adjusted to Find entities that satisfy the specifications provided. And a specification is just a wrapper around a predicate. Oh, and a predicate is just a fancy word for a condition. For example, consider this code:
if (a < 3) a++;
The part that says “a < 3” is the predicate. We can effectively change that same code to the following:
Func<int, bool> predicate = someInt => someInt < 3;if (predicate(a)) a++;It might seem like code bloat in such a simple example, but the ability to reuse a ‘condition’ to check in many places will be a life saver when your systems start to grow. In our case we care about predicates because LINQ is full of them. For example, the “Where” statement takes a predicate in the form of Func<T, bool> (where T is the generic type on your IEnumerable). In fact, this is the exact reason we are also interested in predicates in our specification. Each specification represents some kind of filter. For example:
Products.Where(x => x.Rowkey == “1”)
The part that says x.Rowkey == “1” is a predicate, and can be made reusable as a specification. You’ll see it in action in the final code below, but for now we’ll move on to our Repository implementation. Just keep in mind that we will be reusing those ‘conditions’ and storing them in their own classes.
We’ll focus first on the definition of the repository class and its constructor:
public class TableRepository<T> : IRepository<T> where T : TableServiceEntity { private readonly string _tableName; private readonly TableServiceContext _dataContext; public TableRepository(string tableName, TableServiceContext dataContext) { _tableName = tableName; _dataContext = dataContext; } // TODO CRUD methods }Our table repository implements our interface and most importantly takes a TableServiceContext as one of its constructor parameters. And to complete the interface contract we must also ensure that all generic types used in this repository inherit from TableServiceEntity. Next we’ll add in the Add/Update/Delete methods since they are the easiest:
public void Add(T item) { _dataContext.AddObject(_tableName, item); } public void Delete(T item) { _dataContext.DeleteObject(item); } public void Update(T item) { _dataContext.UpdateObject(item); }Simple enough, since we have the generic table service context at our disposal. Likewise we can add in the SubmitChanges() method:
public void SubmitChanges() { _dataContext.SaveChanges(); }We could just call SaveChanges whenever we add or delete an item, but this makes it more difficult to do batch operations. For example we might want to add 5 products and then submit them all as one query to the table storage API. This method lets us submit whenever we like, which is keeping with the same approach used when creating your own TableServiceContext or using the default one.
Finally, we need to define our Find method which takes zero or more specifications:
public IEnumerable<T> Find(params Specification<T>[] specifications) { IQueryable<T> query = _dataContext.CreateQuery<T>(_tableName); foreach (var spec in specifications) { query = query.Where(spec.Predicate); } return query.ToArray(); }Every specification must have a predicate (refer to the initial definition and you will see the property is defined as ‘abstract’ which means it must be overridden). And a predicate is a Func<T, bool> and the T type is the same type as our repository. Therefore we can simply chain all the predicates together by calling the .Where() extension method on the query over and over for each specification. At the end of the day the code is really quite small.
And that’s all the framework-like code for setting up the Repository and Specification patterns against table storage. To show you how it works we first need a specification that allows us to get a product back based on its row key. Here’s an example:
public class ByRowKeySpecification : Specification<Product> { private readonly string _rowkey; public ByRowKeySpecification(string rowkey) { _rowkey = rowkey; } public override Expression<Func<Product, bool>> Predicate { get { return p => p.RowKey == _rowkey; } } }In this specification, we take a row key in the constructor, and use that in the predicate that gets returned. The predicate simply says: “For any product, only return those products that have this row key”. We can use this specification along with our repository to perform CRUD operations as follows:
var context = tableClient.GetDataServiceContext(); IRepository<Product> productRepository = new TableRepository<Product>("Products", context); productRepository.Add(testProduct); productRepository.SubmitChanges(); testProduct.Price = 21.99; productRepository.Update(testProduct); productRepository.SubmitChanges(); var byRowkey = new ByRowKeySpecification("1"); var results = productRepository.Find(byRowkey); var result = results.FirstOrDefault(); productRepository.Delete(result); productRepository.SubmitChanges();Tada! We now have a strongly typed repository that will work on any entity type you want to use. And the great thing about repositories is that because we have an IRepository abstraction we can implement an ‘in memory’ version of the repository which is very useful for unit testing.
Summary
As we progressed through the three options the amount of code got larger but I think we also got closer to true object oriented programming by the end there. Personally I like to always use repositories and specifications because it means we can write our code in a way that the persistence mechanism is irrelevant. We could easily decide to move products into our Sql Azure database and instead use a SqlRepository<T> instead of the TableRepository<T>.
Hopefully you’ll find the concept useful and aim to start with scenario 3 in all cases. To help you get started, I’ve assembled all 3 options into a reusable library for you, downloadable from here:
Table Storage Examples Library (163 Kb)
In each of the scenario folders you’ll find a single starter class that inherits from the TableStorageTest abstract class; you can look at that class to work out how the particular scenario works.
In the near future I will be looking to create a number of these basic classes as a reusable library to help Windows Azure developers get up and running faster with their applications. But in the mean time, happy coding.
<Return to section navigation list>
• Dilip Krishnan’s Series On Available Authentication Mechanisms For OData Services And Clients post of 7/24/2010 to the InfoQ blog reviews Alex James’s series of blog posts about OData services authentication techniques:
The WCF Data Services Team have recently been doing a series on the available authentication mechanisms for client/OData service authentication. OData is an implementation of the ATOMPub protocol with extensions to query and update ATOM resources. From the OData website.
Alex James a Program Manager on the Data Services Team provides a series of articles, in an attempt to field authentication related questions.
- How do you ‘tunnel’ authentication over the OData protocol?
- What hooks should I use in the WCF Data Services client and server libraries?
According to Alex the answer lies in specific usage scenarios; each of which addresses a different type of challenge. He frames the answer as a set of questions that provide insights into the appropriate authentication option.
- How does an OData Consumer logon to an OData Producer?
- How does a WCF Data Service impersonate the OData Consumer so database queries run under context of the consumer?
- How do you integrate an OData Consumer connecting with an OAuth aware OData Producer?
- How do you federate a corporate domain with an OData Producer hosted in the cloud, so apps running under a corporate account can access the OData Producer seamlessly?
Here is an overview of the list of scenarios covered in the series
Windows Authentication – Covers authentication using Windows Credentials. This predominantly supports an intranet-enterprise scenario where the network homogeneity only Windows based servers and clients
Custom Basic Authentication – Covers the case when the basic challenge-response authentication at the infrastructure level (IIS) using a username/password is not sufficient; for e.g. if the user/password store is in a database
Forms Authentication - Covers forms authentication where the method of protection is achieved at the framework level in an IIS hosted ASP.net environment.
ClientSide Hooks – Covers client options when Windows and Basic Authentication doesn’t cut it. “Usually applies if you are using a different authentication scheme, for arguments sake OAuth WRAP, the Credentials property is of no use. You have to get back down to the request and massage the headers directly [e.g. when using] Claims Based Authentication.”
Server Side Hooks – Covers the different OData service hosting options IIS , WCF or custom host and explores the different ways to implement authentication under each host.
The team is, of course, looking for feedback on “any Auth scenarios you want [
us] to explore”. If you’re looking to use OData Services to expose data in a RESTful fashion that’s secure as well, be sure to check out the series.
• Chris Sells’ detailed Open Data Protocol by Example white paper from the MSDN Library of March 2010 is probably the best single overview of OData you’ll find. It begins:
Exposing data-based APIs is not something new. The ODBC (Open DataBase Connectivity) API is a cross-platform set of C language functions with data source provider implementations for data sources as wide ranging as SQL Server and Oracle to comma-separated values and Excel files. If you’re a Windows programmer, you may be familiar with OLEDB or ADO.NET, which are COM-based and.NET-based APIs respectively for doing the same thing. And if you’re a Java programmer, you’ll have heard of JDBC. All of these APIs are for doing CRUD across any number of data sources.
Since the world has chosen to keep a large percentage of its data in structured format, whether it’s on a mainframe, a mini or a PC, we have always needed standardized APIs for dealing with data in that format. If the data is relational, the Structured Query Language (SQL) provides a set of operations for querying data as well as updating it, but not all data is relational. Even data that is relational isn’t often exposed for use in processing SQL statements over intranets, let alone internets. The structured data of the world is the crown jewels of our businesses, so as technology moves forward, so must data access technologies. OData is the web-based equivalent of ODBC, OLEDB, ADO.NET and JDBC. And while it’s relatively new, it’s mature enough to be implemented by IBM’s WebSphere[ii], be the protocol of choice for the Open Government Data Initiative[iii] and is supported by Microsoft’s own SharePoint 2010 and WCF Data Services framework[iv]. In addition, it can be consumed by Excel’s PowerPivot, plain vanilla JavaScript and Microsoft’s own Visual Studio development tool.
In a web-based world, OData is the data access API you need to know. …
Chris continues with the paper; it’s longer than this post.
• The Microsoft Project Code-Named “Houston” Services Community Technology Preview Privacy Statement was made final on 7/16/2010:
This pre-release (CTP) of Microsoft Project “Houston” is a database administration tool for SQL Server and SQL Azure users. Users with a SQL Azure account log into Houston by providing their SQL Azure account identity and password and by identifying the database that they will be working with. Houston allows SQL users to develop, deploy, and manage the database. For example, Project “Houston” will make it easier to:
- View statistics about the database and list the objects within the database.
- Author and execute T-SQL queries.
- Design tables and interactively edit table data.
- Create, edit, and execute common database objects including views and stored procedures.
The statement continues with typical sections.
See my Test Drive Project “Houston” CTP1 with SQL Azure updated 7/23/2010 for more information about Project “Houston.”
Doug Rehnstrom’s Planning a Move to the Cloud post of 7/24/2010 to Knowledge Tree’s Cloud Computing blog narrates the steps in moving an SQL Server database to SQL Azure with SQL Azure Data Sync and them migrating data entry operations:
In an earlier post, The Problem with Moving to the Cloud is Everything Works, I wrote about the difficulties of moving a customer’s applications to the cloud. The point was that you need to have a plan to make that transition smooth. Well, we’ve developed at least an initial plan.
First we’re going to use Microsoft Sync Framework to create a SQL Azure version of our database. We’ll also write a .NET program to keep the local database and the SQL Azure version synchronized. At that point everything will keep working as before, and the SQL Azure database will just be a copy.
Step two will be getting our main data entry system working using SQL Azure. This really amounts to changing the connection string. My guess is the program will work fine. Some changes will likely be needed to optimize performance when our data is being accessed over the Internet.
Of course we’ll have to set up a test bed before doing it for real. The cloud will make that simple though.
Next we’ll move the Web apps one by one into Windows Azure. I’m not too worried about that, and to be honest, I don’t care if it takes a while. We always move at a snail’s pace anyway.
Lastly, I want to move file sharing and authentication services into the cloud. I’m not entirely clear about what to do about authenticating users, but I know I can figure that out using the AppFabric.
Then, I’ll turn off the servers and throw them away (well maybe not).
Learning Tree course 2602: Windows® Azure™ Platform Introduction covers each of these technologies; Windows Azure, SQL Azure, Microsoft Sync Framework and the AppFabric. One of the nice aspects of authoring an overview course like this is that it forces me to learn the details of a broad range of technologies and features. This is also exactly what I need for my customer.
I’ll let you know how it goes.
@MS_SQL_Server tweeted on 7/23/2010:
I agree that “it’s not a replacement for SQL Server Management Studio 2008 R2.” See my illustrated Test Drive Project “Houston” CTP1 with SQL Azure post updated 7/23/2010.
Composite C1 claims support for OData via a package called Composite.Tools.OData on this OData page:
Short description
Composite C1 supports OData via a package called Composite.Tools.OData. And thus, you can query the Composite C1 data store using the OData. Data types you define via the UI works as well as the data types that are a part of Composite C1 and installed packages.
How to implement OData support
To start querying the C1 data store using OData:
- Download the Composite.Tools.OData package.
- Install the package on your Composite C1 site as a 'Local package' - you can do this via System | Packages | Installed Packages | Local Packages.
Querying data
Once the package is installed you can query via the OData protocol on the URI the URI as "
http://<hostname>:<port>/OData/OData.svc". On information on what client libraries you can use, please visit the OData SDK page .To get instant LINQ and OData satisfaction, try calling your OData service using LinqPad like this:
- Download, install and run LinqPad (http://www.linqpad.net/)
- In the LinqPad window, click "Add connection".
- In the wizard, select "WCF Data Services" and click "Next".
- In the next window, select "Data Services" for the Provider.
- Type the URI as "
http://<hostname>/OData/OData.svc". (Please replace <hostname> with your hostname.)- Click "OK". LinqPad will populate the right pane with the contents of the selected C1 data store.
- Start writing LINQ...
Technology demo
To get a quick intro to the OData package and examples on how it can expose both native Composite C1 data type and data types you define yourself, watch this video:
Important Notes
- Security is not implemented in the package and it is used solely for technology demonstration.
- Full OData support is not implemented in this demo.
- For easy deployment the OData package will update your web.config and enable WCF services to use multiple site bindings - for more info on this setting, please see Supporting Multiple IIS Site Bindings.
Requirements
Composite C1 version 1.3 or later
Wall-pc posted Sample Databases for SQL Server 2008 R2 (and more) on 7/23/2010:
I updated my development PC to Visual Studio 2010 recently, just prior to the Azure boot camp, and also updated to SQL Server 2008 R2. But, which Sample Databases to use with SQL Server 2008 R2?
Here is a quick list of selected Sample Databases (incomplete, possibly outdated):
AdventureWorks2008R2AZ.zip SQL Azure (December CTP) AdventureWorks2008R2_RTM.exe AdventureWorks 2008R2 RTM SQL2008R2.All_Product_Samples_Without_DBs.x64.msi SQL Server 2008R2 RTM SQL2008R2.All_Product_Samples_Without_DBs.x86.msi SQL Server 2008R2 RTM SQL2008R2.Analysis_Services.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Analysis_Services.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.Data_Access_and_DP.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Data_Access_and_DP.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.Data_Engine_Admin_Script.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Data_Engine_Admin_Script.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.Data_Engine_and_FTS.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Data_Engine_and_FTS.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.Integration_Services.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Integration_Services.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.Reporting_Services.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.Reporting_Services.Samples.x86.msi SQL Server 2008R2 RTM SQL2008R2.XML.Samples.x64.msi SQL Server 2008R2 RTM SQL2008R2.XML.Samples.x86.msi SQL Server 2008R2 RTM
SQL Server 2008 R2 samples can be downloaded from Microsoft SQL Server Community Projects & Samples, if you pick through all the various options. The site lists many more sample databases, too. I prefer a quick list of downloads specifically for SQL Server 2008 R2. So lets save some time with the above list.
What is missing from the Sample Databases? Maybe SQL Server 2008 R2 samples for Team Foundation Suite (TFS)? More PowerShell examples? Besides SQL Server 2008 R2, how about some sample database references for Windows Phone 7 (WP7)? Maybe some new databases to flush out all the features in Entity Framework? Other suggestions?
<rant> WTF? OMG. HOW ABOUT SOME NOSQL DATABASES. FTW. </rant> … just kidding… maybe.
Other news is that Northwind Community Edition appears dead. But, we can still download the original scripts.
Enjoy.
SQL Azure runs a modified version of SQL Server 2008, but I’ve found that you can migrate AdventureWorks2008R2AZ.zip to SQL Azure with the SQL Azure Migration Wizard v3.3.3 or later and SQL Azure Data Sync.
<Return to section navigation list>
No significant articles today.
<Return to section navigation list>
Ryan Dunn and Steve Marx roast Jim Nakashima in the latest 00:24:09 Cloud Cover Episode 20 - Visual Studio Tools with Jim Nakashima video segment of 7/24/2010:
Join Ryan and Steve each week as they cover the Microsoft cloud. You can follow and interact with the show at @cloudcovershow.
In this episode:
- Learn how to use Visual Studio to browse your Windows Azure compute and storage.
- Learn how to deploy your application directly from Visual Studio.
- Learn how to do historical debugging of your Windows Azure application using IntelliTrace.
- Learn about using the SQL Azure Migration Wizard to help make the move to SQL Azure.
Show links:
Dan Fernandez conducts A screenshot tour of the new Channel 9 which will enter test mode next week. Channel 9 content will be covered by the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 United States license.
Rinat Abdullin describes Lokad CQRS - Using Protocol Buffers Serialization for Azure Messages in this detailed 7/24/2010 tutorial:
Lokad CQRS, just like any other Application Engine, can use multiple serialization formats to persist and transfer messages. We've tried various options, starting from the XML serialization and up to BinaryFormatter and WCF Data Contracts with binary encoding.
They all had their own issues. Serialization format that had performed best in our production scenarios is called Protocol Buffers.
Protocol Buffers
Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.
ProtoBuf.NET is a great implementation of ProtoBuf serialization for .NET by Marc Gravel (you probably saw him on the Stack Overflow).
Lokad.CQRS uses ProtoBuf serialization internally for transferring and persisting system message information. ProtoBuf serialization is also the recommended approach for serializing messages in Lokad.CQRS:
- format is extremely compact and fast, better than XML Serialization, Data Contracts or Binary Formatter (see below);
- format is evolution-friendly from the start (renaming, refactoring or evolving messages gets much simpler);
- format is cross-platform.
Here's how the performance looks like, when compared to the other .NET options (details):
However, in Lokad.CQRS you should not worry about these specifics and potential problems that have been taken care of. You just define message contracts with ProtoBuf attributes:
[ProtoContract]
public sealed class PingPongCommand : IMessage
{
[ProtoMember(1)] public int Ball { get; private set; }
[ProtoMember(2)] public string Game { get; private set; }
public PingPongCommand(int ball, string game)
{
Ball = ball;
Game = game;
}
PingPongCommand() { }
}and switch to this serialization in the domain module:
.Domain(d =>
{
// let's use Protocol Buffers!
d.UseProtocolBuffers();
d.InCurrentAssembly();
d.WithDefaultInterfaces();
})More information is available in ProtoBuf in Lokad.CQRS documentation. Additionally, Sample-04 (in the latest Lokad.CQRS code), shows implementation of Ping-Pong scenario with ProtoBuf.
Starting from this sample, we'll be using Protocol Buffers as the default serialization in our samples.
BTW, side effect of using fast and compact ProtoBuf serialization is that it increases overall performance. Smaller messages are less likely to exceed 6144 byte limit of Azure Queues. App Engine handles such messages by saving them in Azure Blob Storage. This essentially allows to persist messages as large as a few GB. Yet, second round-trip to Blob is something that we would want to avoid, if possible. ProtoBuf serialization in Lokad.CQRS helps to significantly improve our chances here.
Arie Jones reported the Latest CRM 4.0 SDK now available in this 7/24/2010 post to the ProgrammersEdge blog:
Now available in the latest SDK is what I believe is one of the most exciting releases fro CRM 4.0 since the launch of Accelerators!
Thanks to contributions from the ADX Studios Team [the SDK] now include[s] the Advanced Developer Extensions. Advanced Developer Extensions for Microsoft Dynamics CRM, also referred to as Microsoft xRM, is a new set of tools included in the Microsoft Dynamics CRM SDK that simplifies the development of Internet-enabled applications that interact with Microsoft Dynamics CRM 4.0. It uses well known ADO.NET technologies. This new toolkit makes it easy for you to build an agile, integrated Web solution!
Advanced Developer Extensions for Microsoft Dynamics CRM supports all Microsoft Dynamics CRM deployment models: On-Premises, Internet-facing deployments (IFDs), and Microsoft Dynamics CRM Online. The SDK download contains everything you need to get started: binaries, tools, samples and documentation.
Now for some cool word bingo on the features! included is:
- An enhanced code generation tool called CrmSvcUtil.exe that generates .NET classes (i.e. Generates a WCF Data Services (Astoria/oData) compatible data context class for managing entities)
- LINQ to CRM
- Portal Integration toolkit
- Connectivity and caching management, which provides improved scalability and application efficiency.
Download it here.
I am downloading it now. [A]s soon as it extracts am going to start playing with these extensions! Hopefully you will see some more detailed posts soon on what’s included and some of the cool things you can build!
The Microsoft Demonstrates Next-Generation CRM Release at Worldwide Partner Conference press release gives further details.
Return to section navigation list>
• Jean Paoli delivered an “Open Cloud, Open Data” keynote to O’Reilly Media’s OSCON 2010 conference on 7/22/2010. Following is the keynote abstract and a 00:13:21 video archive:
The cloud is all about more connectivity – and interoperability is at the heart of that. Organizations around the world are looking at opportunities to leverage a new wave of cloud technologies. New data sets. New computing power. We believe that making it easy to move data in and out of the cloud, and having developers chose their favorite programming language are essential attributes of an open cloud. Interoperability is at the core of our vision of a cloud that can foster the imaginations of developers, unleash their creativity and enable them to build new breakthrough scenarios.
Jean Paoli is General Manager, Interoperability Strategy at Microsoft, and one of the co-creators of the XML 1.0 standard with the World Wide Web Consortium (W3C). He has long been a strong and passionate advocate of XML and open standards. Jean manages the Interoperability Strategy team that coordinates the technical interoperability activities across Microsoft.
Jean jump-started the XML activity in Microsoft. He created and managed the team that delivered msxml, the software that XML-enabled both Internet Explorer and the Windows operating system. Paoli helped architect Office XML support and was instrumental in creating the newest XML Office application, InfoPath.
Jean is the recipient of multiple industry awards for his role in the XML industry, such as PC Magazine, Technical Excellence Award – co-creator of XML (1998), InfoWorld – Top Technology Innovators Award 2003, IDEAlliance – XML Cup (2004).
Jean wrote the Foreword to my Introducing Microsoft Office InfoPath 2003 title for Microsoft Press.
You’ll find more about his keynote to the OSCON 2010 conference in his Interoperability Elements of a Cloud Platform Outlined at OSCON post to the Interoperability @ Microsoft blog of 7/22/2010, which was previously reported here. Following is a partial capture of the landing page for microsoft’s new Cloud Services Interoperability site:
Charles Babcock claimed “Emerging tech always comes with a learning curve. Here are some real-world lessons from early adopters” as a preface to his feature-length Cloud Computing Comes Down To Earth article of 7/24/2010 for InformationWeek:
Businesses are testingcloud computing and, in a few cases, beginning deployment. They're after increased flexibility, agility, and economies of scale, but IT veterans know such gains won't be effortless. There's a steep learning curve with this new computing model.
One of the biggest lessons so far is that it's hard to know precisely what your systems are doing in a public cloud environment. Yes, Amazon.com's CloudWatch and services like it will tell whether your workloads are operating, but they don't tell how well apps are performing, such as if they're choking on I/O overload.
SAP's Marge Breya provides a demonstration of the latest BusinessObjects Explorer On Demand, as well as two brand new products not even in beta yet: Kona (a cloud-based BI tool) and 12 Sprints (a collaborative decision making tool, also in the cloud)
Even when there's an outright failure, it can be hard to get the information you need. When part of an Amazon data center in northern Virginia suffered primary and backup power failures on Dec. 9, it took 34 minutes before the news was posted on Amazon's Service Health Dashboard. Amazon acknowledged the outage and offered updates, but it was up to customers to assess the impact. To know whether their workloads were down, companies had to subscribe to CloudWatch--not everybody does--or a service such as VMware's CloudStatus or Alternate Networks' network monitoring, or check directly if a failover activated backup servers, which Amazon encourages you to set up for each workload.
Cloud computing service providers, like their customers, are in learning mode during this break-in period. They include infrastructure veterans Amazon, Google, IBM, Microsoft, and Terremark; outsourcers such as CSC; telecom giants Verizon and AT&T; and newcomers Cloud.com, Engine Yard, Heroku, and many others. If they're candid, they acknowledge they're venturing into unexplored terrain.
How to seamlessly connect enterprise apps with customers, suppliers and trading partners
Strategic Approach To App Integration
Microsoft, which started charging for its Azure cloud services in February, admits it still has work to do, especially in the area of cloud monitoring tools. Enterprise early adopters are in a position to shape how vendors build out these services. A private cloud appliance recently launched by Microsoft is being co-developed with eBay, which will initially use it for the relatively low-volume Garden by eBay service, where it tests partner's ideas and new applications, and eventually for basic auction services, says VP of technology James Barrese.
Even if companies are only testing cloud services, they should explore the inevitable problems that go with an emerging technology, as well as the potential competitive advantages. InformationWeek sought out early adopters to gauge how they're doing in both respects.
Know What Customers See
Jason Spitkoski, CTO of startup Schedule Bin, has a lot riding on cloud computing, a commitment that was seriously tested last month. Schedule Bin will use public cloud infrastructure to offer online applications that let businesses schedule employee work shifts or track teams as they execute tasks. Spitkoski looked at Amazon's EC2 but opted for Google's App Engine cloud service, which he says is better suited for making changes or adding new apps. Schedule Bin's applications, due to go live this summer, are built in Python, making them a good match for App Engine, which runs Python and Java apps.
Google App Engine has proved a solid platform, though its underlying Datastore system, which provides storage for Web apps, went through a very rocky stretch. In May, Datastore suffered three service interruptions, one lasting 45 minutes. In early June, Google stopped charging for Datastore, acknowledging that, since April, latency in retrieving data had become 2.5 times greater than in January to March.
By mid-June, Spitkoski was worried--Schedule Bin's beta customers had noticed a slowdown. "We have demos with potentially large customers soon, and I'm concerned that the apparent slowness will be brought up," he said in an e-mail interview.
Spitkoski considered how much to talk about the cloud infrastructure with customers; he wanted to put the focus on Schedule Bin's features, not on the cloud service that enabled them, or any problems Google is having. "We want to keep the demo straightforward and to the point, which means we don't want to get into fuzzy discussions about clouds, or what we think Google is thinking," he said.
Given Datastore's performance problems, Spitkoski was on the bubble, contemplating switching cloud providers at a time when most of Schedule Bin's development work was complete. "The timing couldn't be worse if we were forced to suspend our customer efforts and focus instead on switching cloud providers," he said.
Fortunately, the crisis passed. By the end of June, a relieved Spitkoski found that Google had "vastly improved" Datastore's performance. "I'm now completely satisfied with Google's App Engine," he now says.
Satisfied, but wiser about overly relying on a single cloud vendor. "I will always have an eye open for alternatives," Spitkoski says.
Enterprise IT teams could face a similar dilemma--getting deep into cloud development before realizing they bet on the wrong vendor. The better approach is to keep researching alternatives.
On the other hand, it can also make sense to work closely with a cloud services provider. With enterprise customers in particular, there's a chance to mold a cloud offering in a way that suits your company's needs if you have a good relationship with the vendor and the resources to collaborate.
Cloud Is Under Construction
Kelley Blue Book is one of the high-profile customers of Microsoft's Azure cloud service, having tested it as a way of handling traffic spikes on the Blue Book Web site, which offers used-car pricing information. If Azure made it possible for Kelley to abandon its second data center, the company could potentially save at least $100,000 a year, says Andy Lapin, Kelley's director of enterprise architecture.
But Lapin doesn't think he can see clearly enough into how well his applications are running in Azure. Microsoft released monitoring and diagnostic APIs late last year, but workload monitoring systems from Microsoft and other vendors are still lacking, he says. Lapin halted the planned off-loading of Web traffic spikes until better tools come along.
Lapin also warns that some concepts that sound a lot like those used in enterprise IT operations may be different when applied to the cloud. For example, the description of Azure's Table Storage service sounds familiar. The service "offers structured storage in the form of tables," according to Microsoft Developer Network documentation.
When Lapin sees that, he thinks in terms of relational database tables, but that's not how Table Storage works. It "isn't really like a big, flat table," he says. "You really only get indexing on a single column so, while you can query any column, performance optimization is very different from using SQL Server."
Google App Engine's Datastore tables have similar limitations. In general, cloud systems don't do joins across tables or deep indexing, the kind of ad hoc information sorting that relational databases specialize in.
If that drawback sounds like a minor thing, listen to Oliver Jones, an experienced Microsoft .Net developer who writes about coding on his Deeper Design blog. In March, about a month after Azure came out of beta, Jones shared his initial experience with Azure Table Storage: "It looks fairly full featured. However, it is not. At almost every turn I have ended up bashing my head against a Table Storage limitation. Debugging these problems has been a bit of a nightmare." Investigate such warnings before turning your development team loose on it.
One selling point for Azure is the presumed degree to which it will work with Microsoft products already in place. Microsoft says all SQL Server queries will translate to run on SQL Azure, its cloud database. In general, however, be careful about assuming compatibility between existing enterprise systems and those in the cloud.
Beware Assumptions
Innovest Systems is a supplier of software as a service for trust accounting and wealth management firms. It provides online decision-support and accounting for companies with a total of $250 billion in investments, including Mitsubishi UFJ, so availability and reliability of its services are critical.
To deliver its SaaS apps, Innovest previously managed its own hardware in a co-located facility run by an outsourcer, Savvis. Between 2004 and 2008, Innovest migrated its production environments to Savvis-hosted virtual servers. More recently, this platform has evolved into what Savvis calls its Dedicated Symphony cloud service, a form of private cloud computing where servers in an otherwise multitenant cloud are reserved for one customer.
A dedicated cloud made a lot of sense to Ray Umerley, chief security officer at Innovest. "We had always struggled with co-located services. We had to maintain hot backups on standby," he says. "Whenever anything stopped, somebody had to go over there and change a tape or a drive." With the Symphony service, Innovest designed the facilities that it wanted down to the specific policies in the firewall protecting the servers, and Savvis installed them and ensured that they ran.
Over the course of their six-year relationship, Innovest had built a close partnership with Savvis and concluded it could trust private-cloud-style operations to its outsourcing partner. A big step was the move from co-location services, in which Umerley and other technical people had to periodically adjust equipment at the Savvis facility, to letting Savvis technicians take over that function. Teams from the two companies covered myriad operational details so that Innovest could guarantee to its customers that their data was being handled in a way that met strict regulations.
Despite all the preparation, just weeks before the switch-over the teams realized they had overlooked a fundamental detail: Innovest ran Windows applications and Savvis-hosted Windows servers, but the version supported by Savvis was Windows Server 2003, while Innovest apps were still on Windows 2000. With 15 days till deployment, Innovest's IT team swung into high gear and migrated the key applications to Windows 2003.
Both parties knew each other's operations well and thought they were practicing the utmost due diligence as they approached the transfer date. The version of Windows Server involved was something so obvious that everyone assumed it would be the first issue considered, not the last. Since Innovest's launch of Symphony services, everything has run fine, and Umerley gives Savvis high marks for offering visibility into its architecture, engineering, and security.
When making a move into the cloud, "know your provider well," Umerley cautions. That means scrutinizing its security practices, and knowing how the provider keeps its data handling in compliance with regulations that govern your business. Umerley recommends being open and putting the pressure on the vendor to spot potential problems. "Be sure to state: 'Here's what we have. Tell us what we will have to change,'" he says.
Watch Where The Data Goes
Don't necessarily write off cloud computing just because sensitive data is involved. But watch that data closely.
Manpower CIO Denis Edwards is eager for his IT teams to experiment with cloud development, to speed up development and cut costs. But he also has a clear policy about data governance: Developers don't have blanket approval to move data into the cloud.
A project with a certain data set may get the OK through Manpower's data governance process. However, if that project expands to more data, it requires a new approval. Don't let sensitive data creep into the cloud as a project's scope expands.
On the other hand, don't assume that the cloud's a nonstarter just because there's sensitive data involved. Some of the most interesting applications will be those where sensitive data stays on-premises yet gets shared or used in some way through cloud services.
That's happening at Lipix, a nonprofit formed two years ago for the purpose of easing the exchange of information among healthcare providers on Long Island in New York. In one year, CTO Mark Greaker has used CSC's CloudLab to establish a central index of patient records being used at 22 of the 25 competing hospitals in the region. The index now covers about 1 million patient records, but the records stay within the hospitals.
How does that work? Greaker has an edge server in each hospital linked to Lipix's index, which resides on servers in the CSC cloud. Greaker adds a hospital every two to three months to the master index, and when he does, he goes to a CSC portal and commissions a virtual server with the CPU, memory, and storage that the hospital needs. The index tells a doctor where a patient's record is, and the doctor can see a read-only version over a messaging system. About 1,000 of Long Island's physicians are using the system, and Greaker has a $9 million grant to reach another roughly 2,000 within three years. With employees focused on establishing the system, not racking hardware, "I've been surprised how quickly we've been able to design and build it," he says.
Platforms Matter
Russell Taga is a VP of engineering at Howcast, a startup trying to capitalize on the explosion of Internet video by specializing in the "how-to" niche. Howcast keeps its catalog of videos in a cloud service run by Engine Yard; 90% of the content is made by contributors not employed by Howcast. It also links to videos elsewhere on the Web, including YouTube.
Howcast builds Ruby-based apps, which proved to be an important factor in choosing its cloud vendor. The startup's Web applications let people search for, create, and edit videos. For it to succeed, however, Taga believes his firm must make it easier for people to produce and air high-quality how-to videos, so he's focused on developing better Ruby-based online applications to aid amateur video makers.
Engine Yard employs leading Ruby developers such as Yehuda Katz, and Howcast is able to tap into that expertise. Taga's company started out as a Java shop, but found it took too long to build and revise code in Java. Many of his developers were familiar with Ruby on Rails as a framework supported language that allows frequent apps changes. Engine Yard meets the table stakes requirement of a cloud provider: "They're stable and keep us up and running," Taga says. At the same time, "they're in touch with the latest software," serving as a trusted adviser as it pushes Ruby in new directions.
Shape The Cloud's Future
Don't like what you see in the cloud? Change it.
Amazon, Google, Microsoft, and others show a keen desire to address unmet needs. The environment's changing fast. Amazon says new services such as Elastic Block Storage and new types of servers such as Cluster Compute Instance came from feedback from developer customers. In the weeks that we researched this story, App Engine's Datastore problems got ironed out enough that Schedule Bin's Spitkoski went from doubtful about his future with Google to being an enthusiastic endorser of App Engine--though one who now stays open to alternatives.
EBay has just emerged as a strong partner of Microsoft's in shaping an internal cloud appliance suitable for building private clouds. The online auction site is looking several years into the future, toward making its IT infrastructure--which handled about $60 billion of auctions last year--easier to manage and more scalable. EBay wants more cloud-like characteristics in its data centers, so resources can be managed as a unit of pooled, virtualized servers and storage.
Yet eBay doesn't see a public cloud infrastructure as viable for its computing needs in the near term. "A lot of today's [public] cloud isn't capable of operating in the mission-critical space," such as transaction processing, says technology VP Barrese.
With Microsoft as its cloud partner, eBay gets someone else to build that environment, while keeping open the option of hybrid environments--and not necessarily only from Microsoft Azure. "There's a lot of potential for Microsoft to set a cloud standard," says Barrese.
Early cloud implementers, even on a much smaller scale, should heed Barrese's assumption that any cloud supplier is a close business partner, looking for and able to accept direction from the customer. There are a lot of mistakes being made and lessons being learned. Vendors are as new at delivering cloud services as customers are at using them, and may prove surprisingly malleable to committed customers. "This is a journey," says Barrese. "We're still in an early day of cloud computing."
Charles Babcock is editor at large for InformationWeek and author of the book "Management Strategies For The Cloud Revolution."
<Return to section navigation list>
• Michael J. Miller begins his The Changing Cloud Platforms: Amazon, Google, Microsoft, and More article of 7/19/2010 for the Forward Thinking … PC Magazine blog with a cliché:
"Cloud computing" means different things to different people. Some use the term when talking about what we used to call "software as service": applications that are Web-hosted, from webmail to Salesforce.com and beyond. Others use it primarily to mean using publicly available computers, typically on an as-needed basis, instead of buying their own servers.
Still others use it to mean accessing both data and applications from the Web, allowing cross-organization collaboration. And some use it to describe "private clouds" that they are building within their organizations, to make better use of their data centers and network infrastructure, and to assign costs based on usage. In short, the term cloud computing is now so broad that it covers pretty much any way of using the Internet beyond simple browsing.
For me, one of the most interesting things happening in this sphere is the emergence of new platforms for writing and running applications in the cloud. Over the past two years, since I wrote about the emerging platforms of Amazon, Google, and Microsoft, these three vendors have moved in very different directions. After some recent announcements, I thought I'd revisit the topic to look at the state of these platforms. …
Michael continues with analyses of the current status of Amazon Web Services, Rackspace and OpenStack, Salesforce.com, VMware, Force.com and finally Azure:
The other platform that gets a lot of attention is Microsoft's Windows Azure, which has officially been available only since its developer conference last November. Azure is clearly a "platform as a service" offering in that it is a closed platform running Microsoft software and is aimed at developers who use Microsoft's development tools, notably the .NET framework. But it offers pricing based on computer services and storage, much like offerings from Amazon and other cloud infrastructure providers.
The basic platform includes Windows Azure, which offers the computing and the storage; SQL Azure, a cloud-based relational database; and AppFabric (formerly called .NET Services), which includes access-control middleware and a service bus to connect various services, whether built in Azure or outside applications. This month, Microsoft released a new version of AppFabric that supports Flash and Silverlight.
Until now, Azure was available only from Microsoft's own datacenters, but the company just took the first steps towards making Azure available for organizations to deploy within their infrastructures. Microsoft announced the Windows Azure platform appliance, which consists of Windows Azure and SQL Azure on Microsoft-specified hardware. This appliance--which sounds like it's actually a large collection of physical servers--is designed for service providers, governments, and large enterprises.
Note this is very different from letting individual customers set up their own servers to run Azure; Microsoft and its partners will be managing the servers themselves, though companies can host their own data. Initially, Microsoft said Dell, Fujitsu, and HP would all be running such appliances in their own data centers and selling services to their customers, based on this appliance. eBay is also an early customer, using the appliance in its data center. I would expect that over time, this would be made available to more customers, and probably offer tighter links between on-premises and cloud servers.
Azure's initial target seems to be mainly corporate developers, people who already use Microsoft's developer tools, notably Visual Studio and the .NET framework. (Microsoft is also trying to compete with VMware in offering virtualization tools to cloud service providers, but that's another topic.)
Larger service providers such as HP and IBM also have their own cloud offerings, typically aimed at providing customized services to very large corporate accounts. IBM recently announced a new development and test environment on its own cloud. But in general, these tend to be company-specific choices rather than the "self-service" cloud platforms the more general platforms provide.
And I've talked to a number of very large customers who are deploying "private clouds": using their own infrastructures with virtualization and provisioning, as part of efforts to make their data centers more efficient.
Cloud platforms are still emerging, and there are still plenty of issues, from the typical concerns about management and security, to portability of applications and data from one cloud provider to another. But it's clear that cloud platforms and services are getting more mature and more sophisticated at a very rapid clip, and many--if not most--of the developers I know are either using these technologies or are actively considering them.
Michael J. Miller is senior vice president for technology strategy at Ziff Brothers Investments, a private investment firm. Until late 2006, Miller was the Chief Content Officer for Ziff Davis Media, responsible for overseeing the editorial positions of Ziff Davis's magazines, websites, and events.
<Return to section navigation list>
• Anne Taylor’s How to Move Two-Factor Authentication into the Clouds podcast from ComputerWorld for VeriSign (requires web site registration) carries this description:
Two-factor authentication (2FA) delivered via the cloud provides a high level of security without high cost.
This Webcast highlights its many benefits:
- Reduced infrastructure costs – no on premise hardware
- Reduced IT burden – no internal expertise required
- Scalability and flexibility – dial up or dial down as needed
- Improved responsiveness and compliance – outside experts keep up with regulation changes
- Anytime, anywhere access to corporate applications
Andrew Conrad-Murray asks “Is the cloud insecure? Maybe. But that's not the first question IT should ask” as a preface to his Cloud Security: Perception Is Reality article of 7/24/2010 for InformationWeek, which begins:
As a result, this CTO's company, which had deployed its applications on top of Amazon's Web service offering, is bugging out of the public cloud and into a private co-location facility. While he believes his team can configure the Amazon service to be just as secure as the on-site option, and the cloud's low startup costs and rapid deployment benefits are attractive, he had to ask: Could the model cost us business?
No matter how many times public cloud providers assert--often correctly--that data is well-protected on their servers, they just can't shake the insecurity rap. And that means CIOs need to ask not just whether the cloud makes business sense, but whether their customers will see it that way. They may not: Security tops the list of cloud worries in every InformationWeek Analytics cloud survey we've deployed. In our 2010 Cloud GRC Survey of 518 business technology professionals, for example, respondents who use or plan to use these services are more worried about the cloud leaking information than they are about performance, maturity, vendor lock-in, provider viability, or any other concern.
That doesn't mean businesses are shunning the cloud. Of those respondents who do use or plan to use these providers, within the next two years, 20% say up to half of their IT services will come from the cloud; an additional 45% say a quarter of their IT services could be delivered that way. The benefits, such as lower deployment costs and faster time to market, are just too attractive, particularly in today's business climate of stagnant budgets and staffing uncertainty. Still, your customers have legitimate questions about running applications in the cloud, whether on infrastructure-as-a service (IaaS) or platform-as-a-service (PaaS) environments. IT must help the business be prepared with good answers to the two main questions we raise, and others specific to the product. It may make the difference between winning business and losing confidence.
First, customers will look for assurance that an application that runs on PaaS is as secure as an application that runs behind an on-premises firewall. The answer will normally be "No--unless it is." It's an irritating response, but that's because cloud security is frustrating. Here's the breakdown.
A Web application you develop and deploy in a PaaS environment is no more--and no less--secure than a Web app you develop and deploy yourself. The basic principles of secure application development don't change because of the cloud. "Cross-site scripting is still cross-site scripting. There's not much difference whether it's in-house or PaaS," says Brian Chess, chief scientist and co-founder of Fortify Software, an application security testing company. The upshot? Developers must be trained to write secure software, regardless of where that software runs. Applications must be tested regularly to ensure that the inevitable vulnerabilities are found and remediated. Building and running an application on top of Windows Azure, Google App Engine, or Engine Yard doesn't excuse an organization from following these principles. …
Download the entire article as an InformationWeek::Analytics Cloud Computing Brief here.
Stephanie Balaouras posted Building The High-Performance Security Organization to the Forrester blogs on 7/23/2010:
I just completed my second quarter as the Research Director of Forrester’s Security and Risk team. Since no one has removed me from my position, I assume I’m doing an OK job. Q2 was another highly productive quarter for the team. We published 20 reports, ran a security track at Forrester’s IT Forum in Las Vegas and Lisbon, and fielded more than 506 client inquiries.
In April, I discussed the need to focus on the maturity of the security organization itself. I remain convinced that this is the most important priority for security and risk professionals. If we don’t change, we’ll always find ourselves reacting to the next IT shift or business innovation, never predicting or preparing for it ahead of time. It reminds me of the Greek myth of Sisyphus. Sisyphus was a crafty king who earned the wrath of the Gods. For punishment, the Gods forced him to roll a huge boulder up a steep hill, only to watch it roll back down just before he reached the top — requiring him to begin again. Gods tend to be an unforgiving lot, so Sisyphus has to repeat this process for the rest of eternity.
If my protestations don’t convince you, perhaps some data will. The following are the top five Forrester reports read by security and risk professionals in Q2:
- Security Organization 2.0: Building A Robust Security Organization
- Twelve Recommendations For Your 2010 Information Security Strategy
- CISO Handbook: Presenting To The Board
- SOC 2.0: Virtualizing Security Operations
- Market Overview: Managed Security Services
These reports focus on overall information security and risk strategy, the structure of the security organization itself, and the redesigning of traditional sec
![clip_image001[6]](http://blogs.msdn.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-99-37-metablogapi/0827.clip_5F00_image0016_5F00_69D75C2C.png "clip_image001[6]")
