|HCS Consulting Group.|
|We make Complex Systems simple|
By Albert D. Kallal
Saturday, August 09, 2003
Using a wan with ms-access. - a instant network primer.
The JET database engine that ships with ms-access is remarkable little file based engine. In applications where you deal with small files in the 50,000 record range, and say 50 tables (and a good portion of them are related to each other!), and say about 5 to 10 users, the little engine just screams in terms of performance (we are talking about a typical standard cheap office LAN). With a well designed application, I find it HARD to create forms that take more then 1 second to load. I have on more then one occasion replaced mini computer systems with an ms-access on a file share for 5 or 6 users, and the resulting performance on these small tables of 50,000 records is simply amazing. Of course, tables with 50,000 records and only 5 or 6 users is not much these days, but that little JET engine really does perform well. However, what about using JET on a WAN?
It is amazing how many companies start deploying a WAN without having asked some real basic questions about how the wan will function. When we talk about a typical office network, we are referring to what is called a local area network (LAN). A WAN is a wide area network. By wide area, we typically mean different buildings, or even employees that want to work at home (tele-commute), or even just a salesman on the road. Often, some companies will actually connect branch offices via cheap high-speed internet. The cost savings of using the internet to build a WAN can be incredible. I seen companies get rid of huge high priced leased lines and save thousands in monthly connection fees. Using the internet to build a wan is usually referred to as a VPN (virtual private network).
The only other question we need to know here is how fast is the connection?
Before I give any more details here, the above is about all you need to know about networking. Save the rest for the tech guy. I don't support, or setup networks. I also don't work on my own car. However, I know about brakes, oil and the kind of gas I am supposed to put in my car. When planning a trip, I do look up the distance I am going to travel. I think we all agree to not look at the distance we plan to travel would be silly.
So, how fast is the network? Did you bother to ask?
A typical office network is rated at 100MBits. In fact, I really don't care, or even want to get into a whole bunch of talk about bits, bytes and kinds of mumbo jumbo. Really, all we really need here is some sense of scale, and the differences in speed between network connections that we plan to use.
So, your office network has a rating of 100
A typical high speed internet connection that we can use to build our WAN has about .7 to 1.5 mbit rating
So, lets take the middle rating of 1. That means that our high speed internet connection is 100 TIMES SLOWER then the office network. Thus, your typical WAN built around a high speed internet connection (VPN) is about 100 times slower then that cheap office network.
And, all those cool commercials about how fast high speed internet is compared to phone lines are correct. A 56k dial up line is rated at .056. Actually, the phone line is closer to about .04 (you never get an actual 56k connection). So, high speed internet is about 1/.04 = 25 times faster then phone dialup. And, yes those TV commercials are about right when they say they are about 30 times faster then your dialup line. So, we have;
Office network = 100
High speed internet = 1
Dialup modem = .04
Ok, so do you think ms-access is going to work over a dial up line? Often, you will find that ms-access is slow with a office network (the 100 guy). So, if something takes 4 seconds on your office network, it will take 400 seconds on your WAN. 400 seconds = MORE THEN SIX minutes! On the dialup line, you get 100 / .04 = 2500 seconds, or 41 minutes.
This means that ms-access typically will not run across a WAN.
We are NOT talking about some network fix here that can be overcome by a 30% increase in speed. Even doubling your speed by increasing the speed of the wan by 2 times, or re-writing the software to use 1/2 the bandwidth is NOT EVEN CLOSE AND IN THE SAME BALL PARK of your office LAN!
When you walk into a sporting goods store and ask the sales man for a nice little inflatable boat, so far that is ok. However, when you tell the sales man that you are about to visit your family that lives across the ocean and the boat is your means of transportation then we have a problem. Some one along the way needs to mention that rowing a small rubber boat across the ocean is not reasonable. Of course, the difference in speed in your wan vs. the LAN is the same thing. So, while most would laugh at the sheer folly of rowing a rubber boat across the ocean, the same applies to trying to use a wan that is 100 times slower. How can a company start to deploy a wan without asking what the current speed is required for running existing applications? The fact that I seen so many questions in newsgroups about why the wan is slow is why I wrote this article.
In my opinion, any network admin who deploys a wan without first gathering some VERY SIMPLE and BASIC information about the existing network applications should get a real earful (perhaps termination is too harsh, but if you are paying for outside tech help to set this network up, and the recommendation is coming *from* your support company, or the ms-access developers), then I would get a refund on time billed to set this stuff up. To have a company spend real support dollars on a setting up a wan with the pre-text that everyone with high speed internet will have a good usable remote connection is really wrong. In other words, if you are reading this article right now, you probably have someone who has already failed to ask some basic questions about the network and its speed. I guess the real question is who is at fault when this kind of thing happens. The same goes for the sporting goods store when they sell you a boat? Should the store tell you it is a bad idea to cross the ocean in the little rubber boat, or perhaps do we need a warning label on the boat? Where I come from, this stuff is NOT allowed to happen when a boat or a network is being setup!
So, here is the ms-access question we are really asking:
Why is ms-access slower on a network connection that is also 100 times slower?
Because 100 times slower is a lot slower!
There should be no surprise here.
Companies like Microsoft when hiring people actually ask questions like:
How would you move Mount Fuji?
The reason why the above is asked is that companies need people with a sense of scale. How big, how far, how much! We are not talking about a real complex software algorithm or even some type of complex networking setup or protocol. We are simply asking what is the speed of the wan compared to the office network. It is a dead simple question, but I continue to be absolute stunned when networks are deployed, and basic questions about speed are not asked!
In fact, a typical ocean liner can make the ocean voyage in about 6 days. So, 100 times slower means that we are talking about 600 days. That is well over one year. In fact, some single personal boats have ROWED across the ocean in about 70 days. While the row boat was a small 20ft boat, the scales of a factor of 100 times starts to really give one a sense of scale here. Hence, trying to row across the ocean is MORE REASONABLE then tying to use access on a wan that is 100 times slower!
In fact, the guy loaded up the row boat with lots of beer and milk. Hum, not a bad idea at all! (And this is a true story!).
This network story gets worse!
If all of a sudden we had a super net (a really high speed internet) that some are now starting to experience, then could you still use ms-access in a file share across a wan? The answer is no, because WANS' are subject to minor and temporary disconnects. Ms-access when used in a file share mode DOES NOT take well to any interruption in the connection. In fact, the result is usually a corrupted file (you will have to go to the previous days backup to restore the data). So, one bad connection can blow out the whole mdb file. Since a bad connection can damage the file, even if we did get the speed out of a VPN, the risk factor would be so high as to again make this whole idea useless. This also why you can't risk running ms-access in a file share mode on a wireless network either!
I am not sure if you are a science fiction fan and seen movies like "The Fly", or Star Trek. In these movies they have a device called a transporter. What happens is that you have a device that can re-assemble the person in a another location. This concept allows you to electronically transport a person to another location. If during transport the connection is broken, then you have parts of transfer on one side, and other parts on the other side. The result is a broken transport. You simply loose part of a complex jigsaw puzzle, and can't re-assemble it it. Access and a JET file share is much the same as that transporter since pieces of the file are transferred over the network and used remotely. If those pieces can't make it back over the network, then you have a broken humpty dumpty that all the kings men can't put back together.
When you run access on a SINGLE computer the mdb file is transferred from the hard disk to the computers temporary working memory (ram). This is a normal process, and is usually quite reliable. There is NO network involved here. You have the following:
In the above you see the access file cusotmers.mdb on the hard drive. When you request a record in access ONLY PARTS OF THE FILE ARE loaded into the computers memory. This is done for reasons of performance. Despite popular rumors and miss-conceptions about a JET database file, the whole table is NOT transferred to memory. This process is quite different then word, or Excel where the WHOLE file is loaded from disk to ram. Since ms-access is a database, and designed for maximized performance, then ONLY the parts of the file are loaded into ram. This is why you can retrieve ONE customer record, and if the file has 100, or 100,000 records, the time to find and load this record into the ram is INSTANT.
Remember, if you pull the plug on your computer when editing a word document, then the document NEVER makes the trip back from temporary RAM to the hard disk. If your computer freezes, or hangs up, then again the word document is not saved. In ms-access, a computer reset, or power off or freeze up is much more serous. The reason is that parts of the file are now on hard disk, and some parts are in the computers ram memory.
Hence, after loading some records from the disk into ram, you have the following setup:
At this point, if you turn off the computer, or break the connection from the ram to the hard disk, then you have a file that is in pieces and cannot be re-assembled. The file is in a very dangerous state, and a system failure will means that those parts in ram will NOT MAKE IT BACK to the disk drive. When you have network, this means you are actually sharing the disk drive ACROSS the network connection. You thus have:
So, now you have a fragile connection from the remote disk drive over the network to YOUR computer's ram. Now, your computer MUST ensure that changes and those pieces make it BACK to the hard disk. If you power off your computer, or BREAK the network connection, then those pieces on your computer will not make the trip back over the network to the disk drive. It is MUCH more easy to break a connection between two computers on a network then from your hard disk to your ram. Thus, a connection from your hard disk to your computers memory is quite reliable. However, if you share the file on a network, then OTHER computers are pulling parts of the data file into THEIR ram (memory). Any break in the connection between the two computers means those pieces never make it back to the file the computer is on (this is just as bad as turning off the computer). The instant the connection is broken then the file can not be re-assembled and you have a corrupted file. The file can't be re-assembled, since pieces are in your computers memory.
It needs pointing out that when you use a true database server, the pieces and parts of the data file NEVER leave the sever machine (they stay in the RAM on the server box). So, data is sent over the connection, but NEVER pieces and parts of the file. Thus while ordering a book on www.amazon.com means that if you turn off your computer...your order will not be completed, but parts of the computer FILES on the amazon server machine NEVER were placed on your machine (those parts never leave the server system). It is these pieces of the file" that you must eliminate when using a WAN or wireless network. This is also the difference between a file share and a true database server like sql server. Sql server will not corrupt when you break the connection because those parts of the file NEVER leave the server.
So, what can you do for WAN? After all many companies do have remote use of their applications, and how do they do this?
There are a few solutions what will work well in a lower bandwidth environments with ms-access.
Use a true client to server setup.
You can move the data store from ms-access to sql server, and then optimize your access
application. MS access is a great front end to sql server. In fact, ms-access is NOT a database, but only a client to your database engine of choice. In fact, ms-access is really just a nice development tool to build applications with. The office cd does come with two database engines that you can use with ms-access. One is JET. JET is the default engine for ms-access, and it is not a client to sever based system. It is what we call a file share. It is this file share that you need to eliminate over the WAN (you must ELIMINATE the pieces of the file being transferred). The other data engine on the CD is a free 100% compatible sql server based engine. This server engine is called the desktop server, or MSDE (Microsoft Data Engine). This engine is the same as sql server, and it is free if you have the office CD. It means that ms-access will be running in a true client to server mode. You can even use the Enterprise manger tools from sql server to run and mange this little engine. It really is the same as sql server, but is throttled for 5 or less users. Some people even have claimed to run 25 or more with this engine by using programming tricks. Regardless, if you outgrow the small engine, then it is a transparent process to switch the data and code to sql server.
So, for low bandwidth applications, you might consider using the sql server engine (either sql server, or the desktop engine that ships on the office CD. This engine has been on that CD for the last two versions, and we already beta testing office 2003. I am mentioning this fact since often people don't realize that ms-access is now a true client to server product. This just simply means that you can use ms-access to create secure, robust and scalable applications for the corporate world. There are companies now running ms-access applications with 1000 user seat counts. Really, the only limitation of ms-access is going to be how much the server database can handle. This means that ms-access has come a long way and has grown up from being just a pc based database. I would even bet that ms-access is the most popular client to Oracle next to Oracles own products!
With good ms-access designs, then using sql server can work in a high speed net connection with ease. That means ms-access will work well with just a speed rating of 1. Some even have had success with using dialup lines to sql server.
Consider a Web based system.
Another possible solution to limited bandwidth is to turn the application into a web based system. However, this really means that you are not using ms-access at all anymore. In fact, you are talking about re-writing the application and setting up a web server.
Consider using Replication.
Replication is a process where two disconnected databases can connect to each other and exchange data. This type of setup can work very well for salesman to take orders with a notebook. When the sales people get back to the office, they then synchronize with the database on the server. All new records added by any salesman will sync into the main databases. Further, the reverse also happens, and thus all new data records will eventually find their way to each notebook. Thus, replication is a good solution when having a constant connection is not possible. Replication can also work over a WAN/VPN setup, but you are exposing your self to possible corruption and damage if the connection is not reliable.
Use Thin Client Technology
If you need to run an EXISTING ms-access application over limited bandwidth, then Terminal Services from Microsoft is another real winner. It simply rocks! TS is a thin client technology. Thin Client simply means that all processing occurs on the server, and only screen updates and mouse clicks are sent down the wire. The beauty of this system is that it essentially works like a web based system. The difference between TS and web based system is that TS does not use HTML, but a propriety and MORE efficient protocol. You can even run TS inside of a web browser. For remote users, it is fabulous technology. In fact, TS is really a remote control technology. The remote support built into windows XP is based on the same technology that TS uses (Remote Desktop Protocol). This is one of my favorite choices for remote use of ms-access since you don't have to re-write your existing application. In other words, you can deploy and use your existing ms-access applications for remote use anywhere in the country and not have to re-write or change one thing in your application.
More info in TS can be found at:
When you have a minute, you can also read my thoughts on thin client technology at:
I hope this article sparks you to simply ask a few questions to the network person as to how fast the WAN is compared to the office network. I also hope the above article will help those budding ms-access developers to avoid some embarrassment when deploying an access application across a wan.
Time marches on and obviously we are not the only ones that want connectivity everywhere. One of the great new features for office 2007 is that MS access can be used with a new system called SharePoint. SharePoint is Microsoft's road and plan to allow all products in the office suite to work seamlessly when people are connected to the Internet (this is called "Cloud" computing in the news media today). In very simple terms this means is you can develop applications in MS access with the tables (data) placed on SharePoint. MS Access works well with SharePoint even in cases where you have limited bandwidth or connections such as wireless that can often break.
Perhaps even more amazing about SharePoint is you can use the free "on-line" edition of SharePoint called Office Live (www.officelive.com). The beauty of using cloud computing means that you don't have to purchase a server, you don't have to "pay" someone to setup a server. You don't have to pay to install software like SQL server. You don't have to purchase anything at all. You simply just run on down to the office live web site and sign up. With this cloud version of SharePoint you simply install your Access application on each user's computer. Those user's can be anywhere in the world with any type of high speed Internet such as wireless at an Internet cafe and you all will be sharing your data. Of course if your Company already has SharePoint running at the man office then that would likely be the preferred solution over Office Live. However with a free online edition of SharePoint you can not only do this for zero cost but you'll likely be up and running in less time than it takes a phone call to get approval from your own main office to setup yours internal version of SharePoint.
To say that this new SharePoint system opens up new doors and horizons for access developers would be an understatement of the century.
Here's a link to a video showing how this works in action:
SharePoint technology is much based on the same concepts as replication (but you don't have to make any extra effort to set this up when you use access 2007 + SharePoint). So, access 2007 has what is called a "offline mode". This off line mode is much like email programs like outlook mail on your computer. You can edit and read and work with the data (or emails) on your computer without an internet connection. When you get back to the office or back to some civilized place where you have an internet connection, you simply click the online button and all the changes are shuffled back and forth between you and the data on SharePoint.
Without question I'll be writing some articles in the future about SharePoint. Simply put, I seen the light, and for me the new road into Cloud computing is MS access + SharePoint.
Update: April 2, 2010
Access 2010 now has the ability to create web sites. These forms and reports are true browser based applications built in ms-access. These applications built in Access can be consumed by any computer with a browser. This means even Smartphone's, Apple or even Linux computers can now consume applications built in access. Of course building 100% web based applications solves all problems of deployment and connectivity. The only technology required on the client side is a standard browser. No ActiveX or SilverLight or any type special add-in for the browser is required. For the Server Side, SharePoint 2010 and what is called Access Web Services is required.
Here is a video of mine where I demo an access WEB application built 100% using ms-access. Note how at the 1/2 way point in the video I switch to running the application 100% in a web browser.
Outlook style calendar form built in Access - see this running in a web browser:
Access 2010 white paper about manageability and migration:
The Access Show: Publish a Access 2007 database to SharePoint with Access 2010
Albert D. Kallal
Edmonton, Alberta Canada