The Future of HyperNews

Help is invited and greatly appreciated. However, if you plan to do a significant modification that you would like me to incorporate into the base distribution, please try to discuss it with me first so we can coordinate the eventual integration.

If you would like to fund a particular development effort, either one I have described, or any other that you might have in mind, please do contact me.

Why is HyperNews Worth Developing?

Information Organization

We are starting to see variations on the HyperNews theme. Anyone can get the HyperNews source and start modifying it to give a different presentation for their forums and messages. The modified source can, in turn, be modified more by others, etc. This chaotic evolution with competition for readership should result in survival of the fittest. I will be happy to have contributed even if my initial HyperNews scheme ends up being mutated beyond recognition.

If HyperNews gets used a lot, the web of information and discussion will evolve along with the presentation of it. I expect there will be many entry points into the HyperNews web via different organizations of information and home pages, and no particular top node. Some parts of the web will be predominately information while other parts will have lots of discussion. Most discussion nodes should have at least some information associated with them. This is beginning to happen as evidenced by the growing number of HyperNews forums on other sites.

Message Archiving

Even with archiving, it is occasionally appropriate to delete messages, move them to a different forum, or expire them after a certain age or event. HyperNews supports such modification of forums and messages via its administrative tools, although more work is needed to make it more convenient. Automatic reorganization is very difficult to do well, but it would be very valuable since manual reorganization is frequently too much work no matter how easy it is.

Automatic searching is also essential with any sufficiently large forum to avoid having to manually browse through the archives. Searching also serves to keep the forum smaller, ironically enough, by helping users find previous threads of interest rather than encouraging them to post new messages about the same old subjects. Searching can also be used to create virtual forums that are identified by the keywords (or the query in general) used to find a collection of messages. In fact, searching could be the means by which forums may be automatically reorganized into a set of virtual forums that are of long-term interest to the community as well as the host.

Sites such as DejaNews and Reference.com provide large-scale archiving of many usenet groups and mailing lists, and many other sites archive particular groups and lists as well as providing a web-based posting mechanism. As the web continues to grow along with the number of users, I believe these centralized message archive services will need become replicated and decentralized as usenet is now. But they also will need to become specialized to various communities as email lists and web-based forums are now.

How to Find New Messages

To help find messages that are new or changed, an alternative presentation could be generated by HyperNews using chronological order of creation or last-modified date. You, the reader, would still have to scroll through the list looking for the point at which you last read things, since each person comes into the system at a different time. This chronological ordering combined with chronological subdivision is also useful for finding older messages created during a certain date period.

A frequently requested feature, in addition to the chronological ordering, is to have HyperNews remember which messages each reader has read, or remember just the last date that messages were read within each forum, and only display the new (or changed) messages. Doing this per-reader customization on the server would clearly be unscalable for a large number of readers, but it would be OK if the number were sufficiently small. If we do this, we also need to support replication of HyperNews forums across many servers so readers could use a nearby server, similar to what is done with Usenet news servers for all newsgroups.

This per-reader memory of what the reader has read is starting to look an awful lot like news reader. Instead of making HyperNews into a usenet-like reader of web-base forums, how about use existing news readers to read HyperNews messages that are distributed via usenet? For more on that, see the discussion on the issue of HyperNews Usenet Gateways.

Another alternative is to deliver new or changed messages to subscribers by email, as HyperNews currently supports with its two-way email gateway. Notification via a mailing list for each forum seems like it is moving backwards, but email turns out to be appropriate technology in this case. Email delivery via a single mail server would be just as unscalable as the web-based per-reader customization except for a couple of facts. First, the number of subscribers is usually much smaller than the number of casual readers. Second, if web-based readers repeatedly poll HyperNews servers for new messages when there are usually not new messages, this is worse for the server than simply delivering new messages when they actually are posted. Finally, each new message can be delivered in the same way to all subscribers, thus enabling scalable delivery by grouping addresses near each other. In fact, this group delivery mode is one step along the path toward the ideal, discussed in A Protocol for Scalable Group and Public Annotations presented at the Spring 1995 WWW conference.

Object-Oriented Redesign

Here are a few of the most important modules and systems I am considering using, in no particular order. Some are already being used.

• CGI.pm and libwww-perl - obviously.
• Apache modperl - allow HyperNews 'get' to be built-in to the server. Another alternative is fastcgi. Most of the HyperNews commands don't need to be optimized in this way because they are used very infrequently.
• CGI-Cache - Frequently used HyperNews pages that take time to generate could be cached. HyperNews may also fetch remote pages for use as the header, footer, or main message of forums, and these should be cached. HyperNews works with caching servers such as Squid.
• ePerl is a Perl template system that permits compilation and use with HTML. Each HyperNews program that generates HTML output would do so via a set of ePerl templates (or bristled files). This would permit easier customization and translation to other languages.
• Majordomo 2 - HyperNews' interface to email is primitive at best. The management of subscription lists and handling of incoming and outgoing mail could be better done by a package such as Majordomo. MHonArc also has advantages but it is not OO yet. MailTools and MIME-tools ought to be involved.
• Getopt::Declare is a command line option processor. Appears to be the most capable of the many alternatives.
• Date::Manip is a date processor. Appears to be the most capable, though admittedly not the fastest.
• Persistent Storage - database and/or filesystem storage of messages and metadata, perhaps using a Tie interface.
• A Collection module - Forums and Groups are collections composed of messages and members respectively. If a nice collection abstraction is not available, I'll have to develop one.
• Some XML parser/generator for data storage and exchange.

A note about Java. We have looked into developing a HyperNews-like system in Java (on the server side) and there are some significant advantages, but one important reason not to do so is, ironically, lack of portability. It is easier to get a Perl program running under CGI on an arbitrary server than installing a Java program with the Java runtime environment. I am sure that will change at some point. The same story is true of many other possible languages, but not all languages will be equally successful.

Forums about Features needing Development

• Threads and Reading Messages - message outline and embedding, navigation, etc. Always need to make this easier, especially for naive users.

• Bidirectional Email Gateway and Subscription - replace with Majordomo 2?

• Usenet Gateways

• Security Issues regarding HyperNews. It should be possible to share authentication credentials across servers, but we must be very cautious.

• Structured Dialogs - constraints on the form of discourse are useful in many cases.

• Searching becomes essential with any significantly large, successful forum.

External Forums Discussing HyperNews Features

• A free-for-all for the School of Information and Library Studies at University of Michigan.
• Desired Feature List for NCSA Federal Workplace use of HyperNews
• Test forum at umd.umich.edu
• Test forum at stthomasu.ca

1 The tree of responses should be (optionally) moderated by the author by Jose I. Icaza, 1995, May 19

1 Moderation...NOT!!!! by isal@egnatia.ee.auth.gr, 1995, Aug 28

2 Moderation in levels by jeffh@ee.pdx.edu, 1995, Dec 27

1 I almost agree by isal@amphipolis.ee.auth.gr, 1996, Jan 23

...

3 Funny you should say that... (Proposal for journal review using HyperNews) by S.Buckingham.Shum@open.ac.uk, 1996, Jun 20

2 Expiry option... by Kris Benson, 1995, Jun 19

1 Oldest mails should be black.New mails should be white.(and use the rest colors too) by isal@egnatia.ee.auth.gr, 1995, Aug 28

Let me explain this color clock better... by isal@egnatia.ee.auth.gr, 1995, Aug 30

4 Link or button to edit from preview page by Daniel LaLiberte, 1995, Jun 26

1 answer by brownem@rings.doc.stu.mmu.ac.uk, 1997, Mar 02

2 Bars and Size of characters Java Applet minihights of a cell. by greek219@swbell.net, 1998, Mar 25

7 copy and move operations need larger box for the urls by John Wells, 1995, Jul 22

9 (Moved) Email interface to HyperNews by Joseph Wang, 1995, Aug 07

11 Group variable in hnrc by A.S. Baranski, 1995, Aug 08

13 "Real" crossposting and multiple base documents by Carlos A. Varela, 1995, Aug 15

1 n-body.html When? by cvarela@isr.co.jp, 1995, Aug 15

14 HyperNews Features... by Cory West, 1995, Jan 10

1 re: Hypernews Features by jlravitz@mailbox.syr.edu, 1995, Mar 05

dialog size "small" vs "large" = browser dependent by star@starshine.org, 1998, Feb 07

16 Who are the members of every discussion? and some other ideas! by Iasson Salkitzoglou, 1995, Aug 28

1 Thanks for the ideas by liberte@hypernews.org, 1995, Aug 28

1 This conversation is observed by 120 users , 10 hidden ones by isal@egnatia.ee.auth.gr, 1995, Aug 29

...

2 What about readers ratings? Arent we late on that? by unknown@egnatia.ee.auth.gr, 1995, Dec 17

...

19 Making responses temporarily invisible , 1995, Sep 07

1 Interesting Idea by liberte@hypernews.org, 1995, Sep 07

20 Suggestion: new Relation types by Axel Boldt, 1995, Sep 09

21 Layout suggestion for edit-response.pl by Axel Boldt, 1995, Sep 11

1 Alternative idea by Daniel LaLiberte, 1995, Sep 11

1 Default values by Christian.Pineault@ville.sherbrooke.qc.ca, 1995, Sep 21

... 6 Message(s)

2 Need Keywords? by alex@ge.digipress.net, 1995, Oct 09

...

2 Relation icons on the Edit Response page by Ken Tanaka, 1996, Mar 04

1 It does look good with Mac Mosaic 2.0.1 by kht, 1996, Mar 04

...

2 Need radio buttons in table cells too. by liberte@hypernews.org, 1996, Mar 04

... 1 Message(s)

23 "Copy" button unintuitive by Axel Boldt, 1995, Sep 09

1 Is "Move" better? by liberte@hypernews.org, 1995, Sep 09

Yes, but... by boldt@math.ucsb.edu, 1995, Sep 11

Nothing like being obvious by liberte@hypernews.org, 1995, Sep 11

25 Suggestions for edit-article.pl by Axel Boldt, 1995, Sep 12

1 Oops by boldt@emile.math.ucsb.edu, 1995, Sep 12

26 Resource limits by Axel Boldt, 1995, Sep 12

1 Disk full detection by liberte@hypernews.org, 1995, Sep 15

set UID on the Hypernews CGI & use quotas? by peterw@mcs.net, 1996, Jun 13

28 Using HyperNews on a WWW Page Now by J.R. Wilson, 1995, Sep 29

30 E-Mail Notification by Alex Rosen, 1995, Oct 09

1 URL at end of mail by Daniel LaLiberte, 1995, Oct 09

31 Background Colors by Alex Rosen, 1995, Oct 09

32 Info icon by S. Kritikos, 1995, Oct 16

33 Special icon for moderator (related to structured dialog topic) by Dwayne Jacques Fontenot, 1995, Oct 17

34 Expiring responses, archiving expired responses... by Dwayne Jacques Fontenot, 1995, Oct 17

37 Could HyperNews include support for binary attachments? by Matthew C. Reprogle, 1995, Dec 04

1 Great idea! by bridgerj@ncr.disa.mil, 1996, Feb 07

2 This suggestion could be a lot simpler to implement .... by courtejm@boat.bt.com, 1996, Dec 02

1 Redirects and references by liberte@hypernews.org, 1996, Dec 03

...

39 HyperNews inside HyperNews (or how do i know you do not censor your mails in your HyperNews database?) , 1995, Dec 17

1 Posting a message in two or more servers by pressing the same button..Is that possible? , 1996, Jan 23

40 JAVA by bob massey, 1995, Dec 18

1 Using Java is a great idea by liberte@hypernews.org, 1995, Dec 18

Worth giving a try by ajithb@ingenia.com, 1996, Apr 17

41 It's cool, or is it? , 1995, Dec 22

1 Organizing is never-ending by liberte@hypernews.org, 1995, Dec 23

What I'm really up to by tomaht@tkk.utu.fi, 1995, Dec 23

Sounds like Lotus Notes by liberte@hypernews.org, 1995, Dec 23

Pressing on by tomaht@tkk.utu.fi, 1995, Dec 24

Indeed, Pressing on by liberte@hypernews.org, 1995, Dec 24

1 A conclusion for the time being by tomaht@tkk.utu.fi, 1995, Dec 25

...

2 editing is necessary for a discussion about HyperNews code , 1996, Jan 25

...

45 Any Interest in a Windows Based Hypernews Client? by George Lambert, 1996, Jan 09

47 What about multiple headers/footers for different base articles? by Mark Jeftovic, 1996, Jan 22

1 Per base article headers/footers already supported by liberte@hypernews.org, 1996, Jan 23

49 A date would be nice... by Ben Yoshino, 1996, Jan 29

1 I agree by tanaka@ngdc.noaa.gov, 1996, Feb 26

2 Date is optionally available by liberte@hypernews.org, 1996, Feb 26

50 Smart Text vs. Plain Text by Steve Easterbrook, 1996, Feb 05

1 Those users - what'll they think of next? by liberte@hypernews.org, 1996, Feb 05

It's when the users don't think that the trouble starts! by steve@atlantis.ivv.nasa.gov, 1996, Feb 05

"I'm trying to think but nothing happens" - Curly by liberte@hypernews.org, 1996, Feb 05

Maybe Automatic text or standard by kht, 1996, Mar 04

Automatic vs Plain by liberte@hypernews.org, 1996, Mar 04

I like "standard" and "preformated" by steve@atlantis.ivv.nasa.gov, 1996, Mar 04

Alternative by liberte@hypernews.org, 1996, Mar 04

Client-side formatting , 1998, Jun 08

51 Timeline for Future Features by Daniel LaLiberte, 1996, Feb 05

1 what about reader's ratings , 1996, Feb 05

1 Ratings didnt Rate by liberte@hypernews.org, 1996, Feb 06

...

2 readers ratings against search engines by isal@vergina.ee.auth.gr, 1996, Mar 20

...

3 I am searching for rsaref libraries to link hypernews with a "two agency vote protocol" by isal@vergina.eng.auth.gr, 1996, Sep 27

...

4 READERS RATINGS...HERE THEY ARE!!! , 1998, Feb 17

2 Timeline with description by liberte@hypernews.org, 1996, Feb 06

1 NT perl revisited by richh@fourgen.com, 1996, Feb 09

... 3 Message(s)

54 To Do List by Elliotte Rusty Harold, 1995, Apr 17

1 Partly done by liberte@hypernews.org, 1995, Apr 18

55 HyperNew Enrollment CGI script by Kenneth C. Schalk, 1996, Feb 02

56 Binary Attachment Revisited by Paul Prescod, 1996, Feb 28

58 Radio Button Suggestion for edit-member.pl by Ken Tanaka, 1996, Mar 04

1 Do most browsers support multiple form submit buttons? by liberte@hypernews.org, 1996, Mar 04

3 Here's how we customized our installation of HN1.9B5.5 by kht, 1996, Mar 20

59 Using Hypernews with Netscape server? by Dan Cornish, 1996, Mar 17 [Moved]

60 Distributed accounts? by Jay Allen, 1996, Apr 10 [Moved]

61 Text wrap in TEXTAREA fields by Martin Huntley, 1996, Apr 16

1 Not so swift by liberte@hypernews.org, 1996, Apr 16

62 server parse question by Sang Han, 1996, Apr 25

1 wrap=soft no problem by liberte@hypernews.org, 1996, Apr 25

63 Suggestion: FirsaAccess: and Lastaccess: in hnpeople , 1996, May 13 [Moved]

64 Messages : Embed Depth / Outline Depth , 1996, May 18

65 Expiring messages , 1996, May 23

1 Expiration is planned by liberte@hypernews.org, 1996, May 23

Archiving is better that expiration by isal@egnatia.ee.auth.gr, 1998, Sep 11

Expiring is a waste by liberte@hypernews.org, 1998, Sep 11

Let black messages exist under hypernews surface. by isal@egnatia.ee.auth.gr, 1998, Sep 12

Automatic Methods Vs Manual Methods by isal@egnatia.ee.auth.gr, 1998, Sep 14

What about just removing things? by liberte@emancholl.pair.com, 1998, Sep 14

Admins by isal@egnatia.ee.auth.grq, 1998, Sep 15

66 Userfriendly copy/move , 1996, May 23

1 Good idea by liberte@hypernews.org, 1996, May 23

67 $headerText, $footerText equivalents for various other places by Jon Tara, 1996, Jun 03

1 Good idea - not difficult, but not high priority by liberte@hypernews.org, 1996, Jun 04

68 Interface Suggestion , 1996, Jun 04

1 Accenting the message body is a good idea by liberte@hypernews.org, 1996, Jun 06

Can it be customizable? by dan@inchq.com, 1996, Jun 06

Customization is the plan by liberte@hypernews.org, 1996, Jun 06

Here are some nifty public-domain icons by r.stephenson@elsevier.co.uk, 1996, Jun 13

69 Multilanguage and accounts on hypernews. , 1996, Jun 07

1 Per user access info, and templates by liberte@hypernews.org, 1996, Jun 11

1 a java(?) question , 1996, Jun 17

...

2 Templates for multilanguage support by vegardh@gruk.no, 1997, Feb 04

... 1 Message(s)

70 Addition of custom Icons - problems , 1996, Jun 21

1 Need icon size configurations by liberte@hypernews.org, 1996, Jul 01

71 HyperNews with different languages by Bjarte Husebø, 1996, Mar 30

1 Changing language is always difficult by liberte@hypernews.org, 1996, Apr 05

1 Multilingual hypernews? Will it be in 2.0? by seindal@kb.dk, 1996, Jun 24

...

3 How about the last news on this topic by parinov@ieie.nsc.ru, 1997, Sep 25

72 Here's some code for Netscape DBM access files by Jon Tara, 1996, Jul 07

73 Use of Hypernews to Enhance Education by Donna Hardy, 1996, Jul 10

1 Comparison with HyperMail by liberte@hypernews.org, 1996, Jul 10

75 Header and Footer are ignored in most scripts by Guido Leister, 1996, Aug 06

1 Template design will make this easier by liberte@hypernews.org, 1996, Aug 06

76 What 'bout newest messages first? by Haralds Jass, 1996, Aug 13

1 Ok, if you insist by liberte@hypernews.org, 1996, Aug 14

77 What about a port over to Win NT by Christopher, 1996, Aug 14

78 Hypernews for Pentium Server by Bill Henry, 1996, Aug 21

79 New icon name confuses server by Nick Pollitt, 1996, Oct 17

80 File Upload capability by James L. Kelly, 1996, Oct 24

1 Not yet by liberte@hypernews.org, 1996, Oct 24

added but. by jinto@hanmesoft.co.kr, 1997, Feb 20

File uploading could be added by liberte@hypernews.org, 1997, Feb 20

81 Upgrading by Bruno R. Depascale, 1996, Oct 25

1 Should be no problem by liberte@hypernews.org, 1996, Oct 25

82 Structured Consensus-based Discussions by Raymond Lutz, 1996, Nov 01

83 HyperNews as a sign-in guest book! by Ert Dredge, 1996, Nov 02

84 HyperNews Index by Philipp Bachmann, 1996, Nov 04

1 Some kind of index is needed by liberte@hypernews.org, 1996, Nov 04

2 Another index implementation by raines@slac.stanford.edu, 1997, Feb 13

1 ''index'' understates this function by an order of maginitude. by btrembla@is2.dal.ca, 1997, Apr 29

... 2 Message(s)

2 numbering is already done, how about <a name>? by star@starshine.org, 1998, Feb 07

85 remove expire messages by Denis Garon, 1997, Apr 02

86 Thread Weaving by Bruce Gingery, 1997, Apr 17

1 Divergence and Pulling Threads Together by liberte@hypernews.org, 1997, Apr 19

87 Moderator pending.gif and Replacing URC files by Bruce Gingery, 1997, Apr 18

88 Counting messages by Benjamin Kahn, 1997, May 09

1 Counting and dates by liberte@hypernews.org, 1997, May 12

89 How refresh could be made more accurate by Jean-Claude Wippler, 1997, May 21

1 More refresh ideas needed by liberte@hypernews.org, 1997, May 21

1 Re: More refresh ideas needed by fielding@kiwi.ICS.UCI.EDU, 1997, May 21

2 Re: More refresh ideas needed by jcw@meta4.com, 1997, May 21

90 Missing Feature by Paulo Neves, 1997, Jun 19

1 Simpler is sometimes better by liberte@hypernews.org, 1997, Jun 24

simplifing and archiving by neves@inf.puc-rio.br, 1997, Jun 25

archiving is much needed by webmaster@deerfield.net, 1997, Jul 03

Could a cron job do it? by star@starshine.org, 1998, Feb 07

91 Problem with a client-side .hnrc file by Bob Weissman, 1997, Aug 15

1 How about a client-side server? by liberte@hypernews.org, 1997, Aug 18

92 Text Wrapping by Kenny Gardner, 1997, Sep 16

93 localization needed by Franco Bagnoli, 1997, Sep 19

1 localization by .mau., 1997, Oct 22

94 Local Time instead of GMT by Bill Becker, 1998, Jan 14

1 Local time should be possible by liberte@hypernews.org, 1998, Jan 15

Local time: another case for client-side processing , 1998, Jun 09

95 Why WRAP=HARD does not work with IE 3.0 textarea? by Basilius Wibowo, 1998, Jan 21

96 Picklist for subtopic tag, sort by same by Heather Stern, 1998, Feb 07

97 automatically delete outdated messages by Pan Hong, 1998, Feb 11

99 Subscribe users in bluk by Chris 'Floyd' Kreutzer, 1998, Feb 16

100 Subscribe users in bulk by Chris 'Floyd' Kreutzer, 1998, Feb 16

1 Untitled by fngoh@ti.com, 1999, Apr 08

101 Possible to prevent double posting? by Heather Stern, 1998, Feb 28

102 Advertising Banners by Rick, 1998, Mar 30

103 HyperNews on Windows NT by Brian Wilson, 1998, May 15

104 update on annotation support? by Mark Palmer, 1998, May 19

1 Annotation did work, sort of by liberte@emancholl.pair.com, 1998, May 21

Have you seen alexa? , 1998, May 25

105 Interactive stories--an HN variant by Paul A. Clayton, 1998, Jun 08

106 Scroll/Test/Actual , 1998, Jun 20

107 Concerning links on Index page with URL and no-text entries by Paul A. Clayton, 1998, Jul 02

1 OOPS! Without a link how would one make comments? , 1998, Jul 09

2 How some places do it... by davidh+hypernews@psiphi.org, 1998, Aug 06

108 Auth Junk by Bobby Schaetzle, 1999, Feb 18

109 What happens with attatchments? by Quim Gil, 1999, Mar 11

1 Attachments are not handled correctly yet by liberte@hypernews.org, 1999, Mar 12

110 File .responseNum is often lost real count of message and start from zero. by VOG, 1999, Sep 20 [Moved]

111 Spanish version? by Raul Elenes, 1999, Sep 26

1 Templates for alternative languages, styles, etc by liberte@hypernews.org, 1999, Sep 27

112 Job: Research Fellowship to develop 'D3E-HyperNews' by Simon Buckingham Shum, 1999, Sep 27

113 Job: Java Programmer for D3E-HyperNews by Simon Buckingham Shum, 1999, Oct 11

114 What about the forum for developers you have promissed ? , 1999, Nov 07

1 Plans for HyperNews are changing by liberte@hypernews.org, 1999, Nov 07

1 porting hyper news to Java - Some usefull sources by balthasar@glaettli.ch, 1999, Dec 02

... 8 Message(s)

2 ... why not Python, and MetaKit? by jcw@equi4.com, 1999, Dec 22

115 Hypernews: still developing or abandoned? by Richard M. Pavonarius, 2000, Mar 16

1 Still alive by liberte@hypernews.org, 2000, Mar 17

Do not delete old messages, add a date filter instead , 2000, Jun 29

Archiving vs deleting by liberte@hypernews.org, 2000, Jun 29

116 Archive and search?Reality in the future? by Tony Murray, 2000, Jun 21

117 Best version of HyperNews? by Paul T, 2001, May 18

2 1.10 is pretty good by Daniel LaLiberte, 2001, May 21

118 French translation of Hypernews ? by Jean J., 2005, Jan 11

119 SecondLife by Daniel M LaLiberte, 2005, Jun 12

120 Supper by Rapidshare download, Apr 11, 05:40

121 http://Fileshunt.com by Rapidshare download, Apr 11, 05:42

Add	to: "HyperNews Future"

Members

Subscribe

Admin Mode

Show Frames

Help

Web www.HyperNews.org