The Joyent Community

A place where the Joyent community can gather, help each other out, and stay informed.

You are not logged in.

#1 2004-07-11 13:30:43

ryanschwartz
Joyeur Emeritus
From: Madison, Wisconsin
Registered: 2004-06-01
Posts: 1991
Website  Expertise

Alternate web log analysis programs

Please post alternative web log analysis programs in this thread, and we'll assess the issue of installing the best/most appropriate on the system.

Offline

 

#2 2004-07-11 13:36:21

liampage
Member
From: Melbourne, AU
Registered: 2004-06-01
Posts: 122
Expertise

Re: Alternate web log analysis programs

AWStats and Analog appear popular thus far...

Last edited by liampage (2004-07-11 13:36:45)

Offline

 

#3 2004-07-11 13:42:44

nissarup
King of Danish
From: Copenhagen, Denmark.
Registered: 2004-06-02
Posts: 419
Website  Expertise

Re: Alternate web log analysis programs

I have been using phpopentracker at my old host. Not sure if its geared for mass-loggin, though. The database can get quite large.

Offline

 

#4 2004-07-11 14:25:24

enrique
New member
Registered: 2004-06-01
Posts: 17
Expertise

Re: Alternate web log analysis programs

Webalizer is robust and fast.
http://www.mrunix.net/webalizer/

Offline

 

#5 2004-07-11 16:37:58

sderuiter
Member
Registered: 2004-06-01
Posts: 160
Expertise

Re: Alternate web log analysis programs

pphlogger. Also showing 'follow path'. Very useful.

Offline

 

#6 2004-07-11 17:55:37

Alex
Member
From: Providence, RI
Registered: 2004-06-01
Posts: 612
Website  Expertise

Re: Alternate web log analysis programs

What are you looking for in your stats package?

I've taken a look at the above linked packages and I'm not sure which one does the best job for what I want. I doubt it's possible to arrive at a TXD-wide consensus about what we want out of stats, but maybe it will prove useful to look at it functionally.

I have several small sites. Some are personal and some are for business. In both cases I am interested primarily in who came from where and what they looked at (very general I know). Urchin looks very nice, but it seems hard to get at the basic info that I want. There is a lot of navigating involved in getting to the interesting stuff. Is it possible to set it to default another view? The raw number of hits or sessions seems like the least useful information that it offers. I'd love to see the list of IP addresses resolved as domains with their associated number of hits. I'd like it to automatically filter out my IP. Second on my list is what did people look at. This is pretty good, as is, except that I'd like to have it automatically filter out the textpattern directory. It seems odd to me that a system obviously designed with huge numbers in mind requires you to manually filter for each view of data. Am I missing something?

Of the above, I've only used Analog. pphlogger seems close to what I'm looking for. Any thoughts?

Cheers.

Offline

 

#7 2004-07-11 18:44:52

Matt Thomas
Member
From: The country
Registered: 2004-06-02
Posts: 232
Website  Expertise

Re: Alternate web log analysis programs

Urchin looks like a very nice piece of software, but, set out to prove that you can't teach a young dog new tricks, I find myself happy with what I've always used, AWStats. The new version provides some new functionality, but I like it because it's basic. You can set it up to view as a single (long) page that reports the month's visitors, sessions, and hits; broken down by day and hour. It breaks your visitors down by country, gives you your top 10 hosts and robots, average visit duration, top file types, top visited pages, operating systems, browsers, referrers, search keyphrases, errors, and with a bit of included JS, screen sizes and plugin detection.

Urchin can probably do all of that and more, but AWStats is what I know, so I'll probably stick with it. I also like that AWStats can give me up-to-the-minute updates if I choose, so I can see the current day's traffic.

Even with my elementary knowledge of Webmin, I found it pretty easy to set up on TXD.

Offline

 

#8 2004-07-11 19:52:08

jdueck
Lord love a duck
From: Minneapolis, MN
Registered: 2004-06-27
Posts: 968
Website  Expertise

Re: Alternate web log analysis programs

Webalizer is pretty nice. Almost every host I've used since I switched to paid hosting has had it. I find that it gives me exactly what I want to know, broken down by one page per month, which is cool, and a minimum of page views. Urchin has me clicking and downloading more pages just to get all the info I want to see. webalizer shows me what I want to see and no more, it's easy to read and all on one page. Good graphs too.

Just installed shortstats though, I like that pretty well too, since it doesn't index your views of the stats :) If only I could keep it from recording my own hits.

Offline

 

#9 2004-07-12 15:37:41

compooter
hack
Registered: 2004-06-01
Posts: 923
Website  Expertise

Re: Alternate web log analysis programs

I've never been fond of webalizer for it's lack of a cross-referencing breakdown of who was actually viewing what. Analog is OK - I've used it before. I've never been able to set up & use AWStats on any prior host, although I've heard very good things about it and the demo is nice. If I had to make a choice it'd probably be AWStats.

Urchin is pretty darn good. I'm a little upset at Firefox's lack of SVG ability, but meh - whaddyagonnado? Urchin's development around IE/MSFT-based tools is a little annoying. Also - it lacks the ability to use permanent filters, or save filters. But all-in-all it seems like a pretty impressive stats tool.

I don't think the perfect stats package actually can exist. I've always found that a combination of two or three tends to be the best. If I ever need to really investigate mysterious stats, I'll just grep the access_log directly.

Offline

 

#10 2004-07-12 17:14:08

DaveSeidel
Member
From: New Hampshire, US
Registered: 2004-06-01
Posts: 134
Website  Expertise

Re: Alternate web log analysis programs

compooter wrote:

I'm a little upset at Firefox's lack of SVG ability, but meh - whaddyagonnado?

Aha! Turns out this is not a limitation -- all you have to do is install the Adobe plugin and copy a couple of files into Firefox's plugins directory. Works fine for me. Download it here. I can't find the damn page with the instructions for different browsers, but here's what I remember (assuming you're on a Windows box): go to C:\Program Files\Common Files\Adobe\SVG Viewer 3.0\Plugins, and copy both of those files to Firefox. Not officially supported, YMMV, IANAL, yadda yadda yadda.


That gum you like is going to come back in style.

Offline

 

#11 2004-07-12 17:43:02

compooter
hack
Registered: 2004-06-01
Posts: 923
Website  Expertise

Re: Alternate web log analysis programs

Hmm - just gives a fatal error & crashes firefox for me. I think it might need a bit more tweaking in some cases.

Offline

 

#12 2004-07-12 18:05:54

DaveSeidel
Member
From: New Hampshire, US
Registered: 2004-06-01
Posts: 134
Website  Expertise

Re: Alternate web log analysis programs

Sorry about that! What version are you using? I'm on 0.9.1,


That gum you like is going to come back in style.

Offline

 

#13 2004-07-12 18:12:00

compooter
hack
Registered: 2004-06-01
Posts: 923
Website  Expertise

Re: Alternate web log analysis programs

That was via 0.8 - haven't yet tried on 0.9

Offline

 

#14 2004-07-12 19:27:51

reid
Pixel Wrangler
From: Atlanta, Georgia
Registered: 2004-06-01
Posts: 2126
Website  Expertise

Re: Alternate web log analysis programs

Well, I found four files, not two (three dll's and a zip). I copied the dll's to the Firefox directory (0.9.1), but haven't tried to restart the browser yet.

Nice to know adventure could lie ahead...


"Welcome to Hell. Here's your accordion."

Offline

 

#15 2004-07-18 22:46:10

compooter
hack
Registered: 2004-06-01
Posts: 923
Website  Expertise

Re: Alternate web log analysis programs

DaveSeidel wrote:

Sorry about that! ...I'm on 0.9.1...


Upgraded to Fx 0.9.2 -

Offline

 

#16 2004-08-21 09:07:57

mattymcg
Member
From: Melbourne, Australia
Registered: 2004-07-26
Posts: 170
Website  Expertise

Re: Alternate web log analysis programs

Has there been any progress on this? Urchin is magnificent but is only offered to one domain. For those extra domains that one might not be bothered paying for the Urchin license but would still like some basic stats, what lies in store? I noticed the Is Webalizer Enabled? flag in WebMin, is this going to be turned on at some point?

Offline

 

#17 2004-12-02 16:19:58

spiffin
Member
From: London, UK
Registered: 2004-10-19
Posts: 44
Website  Expertise

Re: Alternate web log analysis programs

It's early days .. but this project looks really interesting:

http://das.reinvigorate.net

Offline

 

#18 2004-12-02 22:15:54

Tom
Member
From: London
Registered: 2004-06-08
Posts: 766
Expertise

Re: Alternate web log analysis programs

Urchin sounds like a fab tool.
It would be great to have a dummys guide to getting it work for guys like me.

It looks like a very potent tool, but doctor, what do I do with it?

Reinvigorate is also a great tool. Having been trying it on a bunch of sites and works a treat.


early adopter, late developer ★ VC 200 & Impressionist + Joyous Premium Entrepreneur

Offline

 

#19 2004-12-03 04:02:00

sewm
Member
From: Toronto, Canada
Registered: 2004-06-01
Posts: 233
Website  Expertise

Re: Alternate web log analysis programs

spiffin wrote:

It's early days .. but this project looks really interesting:

http://das.reinvigorate.net

The output of DAS is very nice, and it's actually been around for quite a while but development is slow. I have stats from my site going back to late 2002. I used it up until I switched to TextDrive in June. I especially liked the way it did referrers, much better than Urchin does.

It doesn't actually analyze your logs though. You embed some javascript in your html and the client pings DAS which takes care of all the data collection. There are two drawbacks to this, anyone who has turned off javascript, or is using a browser that doesn't support it, won't get captured by DAS. The other drawback is that the system doesn't track which pages people are actually accessing. You can embed that javascript into every page if you want. But if you want to see which pages on your site are the most popular then you are out of luck. DAS lumps everything together in all the reports. And you'd never be able to tell that that 800k pic of the boss drunk at the Christmas party is eating up all your bandwidth :)

-Sam

Last edited by sewm (2004-12-03 04:03:40)

Offline

 

#20 2004-12-03 05:24:29

noel
All your Noel are belong to us
From: Detroit, MI
Registered: 2004-06-02
Posts: 1184
Website  Expertise

Re: Alternate web log analysis programs

AWStats is my favorite, much like Urchin to me.


noel jackson | http://jcksn.com

Offline

 

#21 2004-12-03 09:43:35

dhdesign
Member
From: off the beaten path
Registered: 2004-06-01
Posts: 422
Expertise

Re: Alternate web log analysis programs

When I moved my site to TextDrive, I started using a new stats program called AudiStat (http://adubus.free.fr/audistat/). It's a very simple PHP/mysql program that tells me who was there, where they went on the site (page by page), how they got there, OS, browser, etc.

All it requires is 1 line of PHP code at the top of each page that you want to track, and voila! you have stats. :)


VC141 | Twitter

Offline

 

#22 2004-12-03 19:20:45

Nukenin
Member
From: USA
Registered: 2004-11-12
Posts: 460
Website  Expertise

Re: Alternate web log analysis programs

My favourite stats program has always been grep. :)

I guess I will start to be concerned about web stats when I start having site visitors (which might happen about the time I have site content, which is some indeterminate time in some indeterminate future).

One of these days I will actually log into urchin to see what it's all about. :) I think at my last provider I used AWstats or the like. I hardly ever checked my stats except when feeling in a particularly "people still visit my site?!?" mood.

As for log/stats programs that require you to modify pages by inserting PHP code, etc, bear in mind that you will want to supplement with a normal log analysis program if your site is heavy on images or sound files or other non-webpage content.


R. Cliff Young  /\/ Rogue Bard
          Chaos Never Died

Offline

 

#23 2005-04-10 19:02:22

aneroid
New member
From: bombay
Registered: 2005-04-07
Posts: 14
Expertise

Re: Alternate web log analysis programs

decided to bump this old thread since it's got most of the alternatives. i'm looking mostly coz urchin is way too slow when it loads. on a per-page basis, it takes way too long. it's also pretty crappily designed for 800x600. there's very little in the left menu, but it's unncessarily wide. i know most ppl think 1024x768 is standard, but seriously...the main area of the page doesn't have so much that the page needs to be so wide.

a good change would be to put everything in one category (traffic, etc.) available on one page...that would compensate for how long it takes. i think in most cases, ppl view more than just bytes transferred or page hits, so putting one on each page doesn't save bandwidth or cpu cycles.

if there was a "light" version or some changes i could make to reduce all the extras, i wouldn't complain.

wrt features, no issues. creating/applying groups could be made a little simpler.

dhdesign wrote:

When I moved my site to TextDrive, I started using a new stats program called AudiStat (http://adubus.free.fr/audistat/). It's a very simple PHP/mysql program that tells me who was there, where they went on the site (page by page), how they got there, OS, browser, etc.

All it requires is 1 line of PHP code at the top of each page that you want to track, and voila! you have stats. :)


phpopentracker works on this basis too which is why i like it. most of the content on my site will be through my scripts so that works just fine. only downside is, that stat collecting does slow things down...if u consider the fact that urchin and awstats use the logs and (in urchin's case, dunno about awstats) is processed just once a day. processing min{1/day, 1/statslogin} would be ideal.

right now, awstats seems to meet my requirements. (details + content). is there anyone who's used both and can give a reason why one might be better than the other. i'm aware ppl enjoy the eye-candy in urchin, but i think it's quite...ugly. that factor aside, anything worth noting in either?

Offline

 

#24 2005-04-10 20:16:00

Tom
Member
From: London
Registered: 2004-06-08
Posts: 766
Expertise

Re: Alternate web log analysis programs

Shaun Inman has got an interesting solution with his ShortStats

I was using DAS-Reinvigorate before, too, but as that is down for the time being ShortStats is working well. It also offers a breakdown of hits by country.

It's perhaps more of the easy solution for too design conscious ping-friends, but might offer insight for other users.


early adopter, late developer ★ VC 200 & Impressionist + Joyous Premium Entrepreneur

Offline

 

#25 2005-04-10 21:14:10

J
Member
From: Paris, France
Registered: 2005-02-24
Posts: 320
Website  Expertise

Re: Alternate web log analysis programs

Urchin has some issues on the hosting sides. I have no specifics, but I've seen a couple of hosting company ranting about it and switching to AWStats.

Offline

 

#26 2005-04-11 06:29:14

rosshill
Member
From: Geelong, Australia
Registered: 2005-02-25
Posts: 67
Website  Expertise

Re: Alternate web log analysis programs

Bump, whats happening ryan?


Thrive Web Marketing - It is time to let your website Thrive!
Ross Hill - Personal site.
CoverHunt - Free cd cover album artwork search.
HatchThat - weekly entrepreneurial interviews.

Offline

 

#27 2005-04-11 07:52:04

jason
a chief (i started this place)
From: San Francisco
Registered: 2004-06-01
Posts: 8821
Website  Expertise

Re: Alternate web log analysis programs

We're logging into dbs, want to do it via spread, and build the stuff worth looking at into textpanel.

Offline

 

#28 2005-04-11 16:06:49

aneroid
New member
From: bombay
Registered: 2005-04-07
Posts: 14
Expertise

Re: Alternate web log analysis programs

not being pushy, but approx. what time frame are we looking at? days? weeks?

i read up on awstats as a webmin module. but i don't see webmin config anywhere, so can't add the module either. so we'll have to wait until it's set up by u guys.

Offline

 

#29 2005-04-11 23:21:39

sunnydays
Southern Sun
From: Adelaide
Registered: 2005-02-23
Posts: 946
Website  Expertise

Re: Alternate web log analysis programs

jason wrote:

We're logging into dbs, want to do it via spread, and build the stuff worth looking at into textpanel.


Awesome!

Offline

 

#30 2005-04-11 23:59:44

aneroid
New member
From: bombay
Registered: 2005-04-07
Posts: 14
Expertise

Re: Alternate web log analysis programs

sidenote/offtopic: should probably be on another thread (but since i made the observation here)...the time zone management isn't working right. 'sunnydays's post says "Today 05:51:39". it's only 05:20 right now and my time zone is set up right (in the profiles).

i thought that punbb isn't handling the +/-0.5 timezones (i.e. it should be 04:51) but the fact that it IS on a +/-0.5, i'm guessing it's coz of DST. since it's "on" right now and the clocks are an hour ahead.

wrt. to GMT, shouldn't the time handling function/code handle places without daylight saving time?

i'd post a but report at PunBB.org Forums Bug reports but no guest posting and i'm not gonna create an account just for that :P

maybe someone will do the needful ;-)

Offline

 

#31 2005-04-12 06:48:09

Geary
Member
Registered: 2005-02-06
Posts: 51
Expertise

Re: Alternate web log analysis programs

aneroid wrote:

i read up on awstats as a webmin module. but i don't see webmin config anywhere, so can't add the module either. so we'll have to wait until it's set up by u guys.


No, you don't need to wait. I installed AWStats in my account here a month or two ago with no problems. I like it much better than Urchin--instead of so many little pages, it puts all the main stats in one big page that I can simply scroll through.

Just follow the instructions in the AWStats distribution and you can be up and running.

Offline

 

#32 2005-04-12 21:47:06

padawan
Moderator
From: Nouméa, New Caledonia
Registered: 2005-02-24
Posts: 387
Website  Expertise

Re: Alternate web log analysis programs

Summary. Used it for years at Capgemini, infinitely more efficient, faster, less resources consuming than equivalent solutions out there. (Summary SP Plus in the case of TxD.)


François
--
My company: http://ubiquitic.com/
My blog: http://padawan.info/

Offline

 

#33 2005-04-12 23:50:27

aneroid
New member
From: bombay
Registered: 2005-04-07
Posts: 14
Expertise

Re: Alternate web log analysis programs

Geary wrote:

I like it much better than Urchin--instead of so many little pages, it puts all the main stats in one big page that I can simply scroll through.


precisely why i want to use it. it also loads alot faster. (it = awstats)

aneroid wrote:

i'm looking mostly coz urchin is way too slow when it loads. on a per-page basis, it takes way too long.
...
a good change would be to put everything in one category (traffic, etc.) available on one page...that would compensate for how long it takes. i think in most cases, ppl view more than just bytes transferred or page hits...


@padawan: is 'summary' worth the price tag? after considering the alternatives?

Offline

 

#34 2005-04-13 07:53:01

padawan
Moderator
From: Nouméa, New Caledonia
Registered: 2005-02-24
Posts: 387
Website  Expertise

Re: Alternate web log analysis programs

aneroid wrote:

@padawan: is 'summary' worth the price tag? after considering the alternatives?


I think so. The demo is fully functional, anyone can give all versions a try for free.

One thing that Summary has and most others have not is a dynamic mode. Instead of spitting out results as static pages (which is there as an option), it computes and keeps metrics in a database. You can make dynamic searches all over the reports, which is very useful when you're tracking something down. But there I reckon it would probably require dedicating one server to crunch and hold the metrics for the whole TxD farm.


François
--
My company: http://ubiquitic.com/
My blog: http://padawan.info/

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2005 Rickard Andersson