Montgomery Chris Contr AFSFC/SFPA 15 February 2005 04:37:20
Howdy,
Our CF log files (application.log, scheduler.log) are huge, in the range of hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS WordPad and MS Word die when trying to open these humongous files.
Is there a text editor or other log viewing tool that can open large log files? What does everyone use?
Thanks.
P.S. Yes, I will be archiving the existing logs and starting new ones and checking them more often.
-- Chris Montgomery, VAMP Programmer HQ AF Security Forces Center, Antiterrorism Branch 1517 Billy Mitchell Blvd, Bldg 954 Lackland AFB, TX 78236-0119 DSN 312.945.7034 Comm 210.925.7034
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
-- Howie Hamlin - inFusion Project Manager On-Line Data Solutions, Inc. - www.CoolFusion.com inFusion Mail Server (iMS) - The Award-winning, Intelligent Mail Server PrismAV - Virus scanning for ColdFusion applications>>> Find out how iMS Stacks up to the competition: http://www.coolfusion.com/imssecomparison.cfm
--- On Monday, February 14, 2005 1:00 PM, Montgomery Chris Contr AFSFC/SFPA scribed: --->
Howdy,>
Our CF log files (application.log, scheduler.log) are huge, in the> range of hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS> WordPad and MS Word die when trying to open these humongous files.>
Is there a text editor or other log viewing tool that can open large> log files? What does everyone use?>
Thanks.>
P.S. Yes, I will be archiving the existing logs and starting new ones> and checking them more often.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
Jim Campbell 14 February 2005 21:08:29 [ permanent link ]
Emacs!
.... what?
Seriously though - on Windows I use UltraEdit for just about everything. It's very fast and I regularly open and search through some similarly massive CF log files with it. I have 512 MB of RAM on my work machine.
Our CF log files (application.log, scheduler.log) are huge, in the range of>hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS WordPad and MS Word>die when trying to open these humongous files.>
Is there a text editor or other log viewing tool that can open large log>files? What does everyone use?>
Thanks.>
P.S. Yes, I will be archiving the existing logs and starting new ones and>checking them more often. >
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
Montgomery Chris Contr AFSFC/SFPA 14 February 2005 21:15:40 [ permanent link ]
I use UltraEdit at home, so I'll look into trying that here at work since I'm already familiar with it. Am also looking into Dave's suggestion on the tail utility. Thanks!
-- Chris Montgomery, VAMP Programmer HQ AF Security Forces Center, Antiterrorism Branch 1517 Billy Mitchell Blvd, Bldg 954 Lackland AFB, TX 78236-0119 DSN 312.945.7034 Comm 210.925.7034
-----Original Message----- From: Jim Campbell [mailto:jim-dy7x29lvcGkAvxtiuMwx3w@public.gmane.org] Sent: Monday, February 14, 2005 12:08 PM To: CF-Talk Subject: Re: Text Editor for Opening Large Log Files
Emacs!
..... what?
Seriously though - on Windows I use UltraEdit for just about everything. It's very fast and I regularly open and search through some similarly massive CF log files with it. I have 512 MB of RAM on my work machine.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application
Our CF log files (application.log, scheduler.log) are huge, > in the range of hundreds of megabytes (173mb, 280mb, 380mb, > etc.). MS WordPad and MS Word die when trying to open these > humongous files.>
Is there a text editor or other log viewing tool that can > open large log files? What does everyone use?
I use tail for these sorts of things. While tail is a common Unix utility, there are free versions for Windows.
Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
It's an inexpensive shareware alternative to QuickView to open all sorts of file types. Once opened in read only mode, you can choose Edit | Edit to copy the files contents into Word or some other program.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
I use tail for these sorts of things. While tail is a common Unix utility,> there are free versions for Windows.
HOORAY! Tail and grep are absolute essentials IMO. Today, for example, I was doing a BULK INSERT of 3.3 million rows into SQL Server. 2 rows failed... one of which was line 336525 or something like that.
"tail +336525 TB902.TXT | more" showed me that line contained someone whose birthday was in 867 AD. Not valid data for a "datetime" data type in SQL 2000.
Also, "tail -1 application.log" is handy for viewing the last line in the application log file.
I think tail is actually included with Windows Server 2003...
Our CF log files (application.log, scheduler.log) are huge, in the range of> hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS WordPad and MS Word> die when trying to open these humongous files.
As a side note, you might considering writing some kind of script to rotate these log files so they don't get that big =)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application
Jordan Michaels 14 February 2005 22:13:06 [ permanent link ]
Rick Root wrote:
Montgomery Chris Contr AFSFC/SFPA wrote:>
Howdy,>>
Our CF log files (application.log, scheduler.log) are huge, in the range of>>hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS WordPad and MS Word>>die when trying to open these humongous files.>>
As a side note, you might considering writing some kind of script to >rotate these log files so they don't get that big =)>
Scripts like this are already included in most Linux distros. They're even installed by default on some... including CentOS 3.4, which is modeled after RHEL 3 and is what Vivio Technologies uses for it's VPS Servers.
I believe the script is simply called "logrotate", but don't quote me on that. ;)
-- Warm regards, Jordan Michaels Vivio Technologies http://www.viviotech.net/ jordan-foDvJEurFp2LZ21kGMrzwg@public.gmane.org
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
It's free/open source and is better than commercial tail for windows programs that I've seen.
Regards,
-- Howie Hamlin - inFusion Project Manager On-Line Data Solutions, Inc. - www.CoolFusion.com inFusion Mail Server (iMS) - The Award-winning, Intelligent Mail Server PrismAV - Virus scanning for ColdFusion applications>>> Find out how iMS Stacks up to the competition: http://www.coolfusion.com/imssecomparison.cfm
--- On Monday, February 14, 2005 1:15 PM, Dave Watts scribed: --->
I use tail for these sorts of things. While tail is a common Unix> utility, > there are free versions for Windows.>
Fig Leaf Software provides the highest caliber vendor-authorized> instruction at our training centers in Washington DC, Atlanta,> Chicago, Baltimore, Northern Virginia, or on-site at your location.> Visit http://training.figleaf.com/ for more information!>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
Greg Morphis 14 February 2005 22:24:31 [ permanent link ]
mTail (tail for windows) craps out on me at above the 500,000 read buffer setting. I use UltraEdit at home.
On Mon, 14 Feb 2005 14:00:47 -0500, Rick Root <rick.root-t5L+qUZdFgkhTx4l/auMhA@public.gmane.org> wrote:> Dave Watts wrote:> >
I use tail for these sorts of things. While tail is a common Unix utility,> > there are free versions for Windows.>
HOORAY! Tail and grep are absolute essentials IMO. Today, for example,> I was doing a BULK INSERT of 3.3 million rows into SQL Server. 2 rows> failed... one of which was line 336525 or something like that.>
"tail +336525 TB902.TXT | more" showed me that line contained someone> whose birthday was in 867 AD. Not valid data for a "datetime" data type> in SQL 2000.>
Also, "tail -1 application.log" is handy for viewing the last line in> the application log file.>
I think tail is actually included with Windows Server 2003...>
Douglas Knudsen 14 February 2005 23:48:05 [ permanent link ]
there are CFAdmin settings for the logs mentioned.
Doug
On Mon, 14 Feb 2005 14:01:53 -0500, Rick Root <rick.root-t5L+qUZdFgkhTx4l/auMhA@public.gmane.org> wrote:> Montgomery Chris Contr AFSFC/SFPA wrote:> > Howdy,> >
Our CF log files (application.log, scheduler.log) are huge, in the range of> > hundreds of megabytes (173mb, 280mb, 380mb, etc.). MS WordPad and MS Word> > die when trying to open these humongous files.>
As a side note, you might considering writing some kind of script to> rotate these log files so they don't get that big =)>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
Mike Chabot 14 February 2005 23:57:49 [ permanent link ]
I use TextPad for my routine log file viewing. I have not used UltraEdit enough to have a good basis for comparison between the two tools. Those two, plus EditPlus, are probably the three most popular Windows tools of this category.
TextEdit has a nice line bookmarking feature. You can search for a particular string, such as a certain Web page, and highlight all of the lines that contain that keyword with bookmarks. You can delete all the bookmarked lines or all the non-bookmaked lines to reduce the amount you have to sift though to find the information you want. For example, if you are trying to trace a common path through the site that led to a particular error, you can bookmark all the lines with ..gif and .jpg in them, then delete those lines (temporarily of course), so you can better focus on the pages that matter.
However, for very large files, such as 1 GB or greater, I am not sure what the best solution is. I had a large Cognos EIS dataset that I needed to edit last week. TextEdit and EditPlus wouldn't even open it. UltraEdit opened it after a long time, but the program was almost completely unresponsive. (I have 1 GB of RAM installed.) I ended up using HJSplit to break the file up, then I edited it in TextPad, then I reversed the split.
I just tried out these tail programs people mentioned. mTail crashed on me after I loaded in too much, but it was quick. Tail for Win32 is nice. It opened the end of the 1.5 GB file very quickly (which was the part I was interested in). However, it seems to lack documentation so I am not sure if it does anything more than view the last X number of bytes is a file. For example, I cannot seem to control how many lines it shows. But it is probably a good solution for monitoring log files. With these other text editing programs, whenever the log file changes in the background, you get alerted and are asked whether or not to reload the entire file, which can be slow for large files over the network. With Tail, you can easily monitor the end of the log file for changes.
If anybody knows of any programs capable of handling 1 GB files with ease, I would be curious if there is anything better than Tail for Win32.
-Mike Chabot
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application
David Lakein 15 February 2005 04:37:20 [ permanent link ]
The issue with opening huge files in text editors is that unless you specifically set an option (e.g., there's a global option in UltraEdit), it keeps a whole undo buffer that uses massive amounts of memory.
FAR Manager http://farmanager.com is a text-mode file+archive manager, that has a very good and fast file viewer. A former coworker swore by it, and I've used it enough to be able to navigate, and find its view command (F3) very useful. (It's got an editor too, but that has too many function commands for me
- David
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67
