=ACTIVITIES OF MATTHEW BISHOP FOR THE ELLA ASSOCIATES ALEXIS PROJECT A Quick Summary And Overview. The majority of the work done by m.j.b can be seen in three directories /var/www/alexis-info/docs /var/www/internal and /var/www/utils All the directories are available from the Web at the urls http://www.ella-associates.org/alexis-info/docs/ http://www.ella-associates.org/utils/ http://www.ella-associates.org/internal/ Also, the directory /usr/local/bin/ contains some general administrative scripts written my m.j.b. This file also gets used as a kind of 'scratch-pad' of ideas which occur to me, and which aren't coherent enough to go anywhere else. Some times you will see a [*] set of characters where they shouldn't be. This is because of my dodgy text to html conversion scripts. Just ignore them. Quite frequently in the text below, I refer to a computer known as the 'Local Server'. This is somewhat confusing since the work 'Local' in this sense does not mean 'local' in the English sense of the work (close by, or not foreign etc) but in the Spanish sense of the word as in 'Shop'. This is because N.Reddel has/ rents a 'space' or 'studio' which in Spanish terms is known as a 'local'. The 'local server' is the server located in this Local which is located in Calle Blanqueria in Barcelona. This computer is running Debian Linux with various Server Applications installed: Apache, Tomcat, MySql, Alexis ... Dates and Descriptions * March 2, 2003 Created Hotmail accounts for testing of James, in particular, the problem concerning the delivery of multiple email's to hotmail accounts. This problem was brought to our attention by Robert Grundy. The hotmail accounts are as follows (e.g. ella1111@hotmail.com) ella1111, ella1357, ella2468, ella3579, ella7531 all accounts have a password of 'bacibaci' It is possible to deliver mail to the accounts using the 'raw' SMTP protocol by connecting to a server like 'mx3.hotmail.com' where the number after the 'mx' part could be 1,2,4 etc, on port 25. The 'Putty' application can be used for this by selecting the 'raw' protocol option. For yahoo you could try 'mx1.mail.yahoo.com'. Specific error messages for James when delivering mail can be found in /usr/bin/james/apps/james/logs/mailet.log An archive containing questions and answers for the James Mail Server can be found at www.mail-archive.com/james-user@jakarta.apache.org An email was sent through James successfully to the hotmail accounts listed above using Outlook express and a comma separated list of email addresses. The answer to the James multiple hotmail delivery problem is possibly that if one incorrect email address is present for a particular destination (eg hotmail) then James fails to deliver any of the email's for that particular destination. However James does successfully deliver mails in the same 'batch' to other destinations (such as yahoo). It is possible that in a sense this is a hotmail problem. For example, when multiple recipients are specified using raw SMTP protocol to somewhere like mx3.hotmail.com, if one of those recipients is not valid, then it appears that none of the recipients will receive the message. According to rfc821 and rfc2821 this appears to be incorrect behaviour. Yahoo Mail for example does not have this problem. It delivers to the valid addresses even if on of the mailboxes is not valid. However, an SMTP mail server could overcome this problem by sending the message to each mailbox individually, which James apparently does not do Wrote small Java program to check hotmail accounts It is located at /root/checkhotmail.class but it only works on Windows (!) Perhaps this is the readLine() method of the BufferedReader class. It is possible that the line separator property for Unix needs to be set in order for the readLine() method to word properly. * March 3, 2003 Researching security problems for the 'local server' Examining the Debian Security How-to/ Manual. There are Debian tools, such as 'nessus' which launch attacks against a Server Computer in order to evaluate how vulnerable that machine is. * March 4, 2003 Not much, thinking about the vi editor. Discovered 'netcat', invoked as 'nc' which is a network analysing tool Which allows scripts to makes TCP connections. * March 5, 2003, Wednesday Working on FTP security: Discovered 'netcat', invoked as 'nc' which is a network analysing tool which allows scripts to makes TCP connections. Have noted an extremely large amount of stuff in the syslogs which looks dubious. It looks roughly like this. -->> Mar 5 19:20:03 www sm-msp-queue[22513]: h21BU1gk005619: to=postmaster, delay=4+06:50:00, xdelay=00:00:00, mailer=relay, pri=55566438, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Mar 5 19:23:01 www /USR/SBIN/CRON[22517]: (mail) CMD ( if [ -x /usr/sbin/exim -a -f /etc/exim/exim.conf ]; then /usr/sbin/exim -q ; fi) --<< It is probably generated by the 'james' mail server but why? Created a directory /var/guest-ftp-setup/ which is a mirror of the /home/vanessa directory which nick created to setup guest ftp. The /var/guest-ftp-setup/ directory has the same permissions and ownerships as the /home/vanessa/ directory created a shell script which automates the process of establishing a guest ftp login account. This script is located at /usr/local/bin/make-ftp-guest and can be invoked as (from any directory) make-ftp-guest theUserName * March 6, 2003, Thursday I am investigating the possibilities of 'sslwrap' to deal with the problem of a secure ftp connection. There is a Debian package for this and reasonably OK information. From reading of the www.wu-ftpd.org mailing list archives it appears that version 2.8 of wu-ftpd will support ssl (or TLS?) but that this is not a stable version yet. The 'local server' is running version 2.6.2 www.landfield.com/wu-ftpd/mail-archive/wuftpd-questions/2002/Aug/0119.html has a list of ftp clients which support secure ftp (ssl/tls) The secure ftp problem is solved very simply; The sshd daemon, which is already running, includes support for the Secure Copy Protocol (SCP). All that is required is a Microsoft-windows client that is able to connect to the sshd server. Two good clients are available. The 'Putty' psftp command line client, and the WinSCP GUI client (which appears to be based on the putty client). The WinSCP client can be found at winscp.vse.cz/eng/ and the putty client can be found at http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html All access to the 'Local Server' as the 'root' user should be made using these clients rather than a standard ftp client connecting to the wu-ftpd server. The normal users of the 'local server' can continue to connect using the 'wu-ftpd' server, since their passwords are not as critical to the security of the system. * March 7, 2003, Friday not much. Am looking at the cvs system, and whether the 'Local Computer' has a cvs server currently running. The Alexis Code needs to be moved into a cvs system. Other things to do: create a swing applet/ application to look up the validity of Hotmail Email Addresses. [note: this was eventually done but not as an applet. It was done as a cgi-script, which is can be found from the URL http://www.ella-associates.org/alexis-info/docs/resources.html ] Type 'info cvs' for the main cvs user documentation or type 'lynx /usr/share/doc/cvs/html-info/cvs.html' for an html version. * March 8, 2003, Inorder not to have to use the -d switch with the cvs checkout command, you need to set the CVSROOT environment variable. The question I have is, where is the best place to set this variable?: In the .bashrc file for each user, in the /etc/profile 'system wide' file, or somewhere else? Summary of importing code into cvs and setting up a module. This process was extracted from the cvs manual (cederqvist). Lines beginning with a 'hash' are comments or non literal intructions. #-- Set up the CVSROOT environment variable so that you don't have to type #-- '-d /var/lib/cvs' every time you use a cvs command. The CVSROOT variable #-- tells the cvs client where it is to find the cvs 'repository', which is #-- where files and their versions histories are kept. CVSROOT=/var/lib/cvs export CVSROOT #-- The above lines only work for the current login session. The #-- CVSROOT environment variable is not set for future sessions. #-- change directory into the top level directory where your code #-- directory structure resides. For example, cd /alexis cvs import -m "Some Comment" directory/path/within/repository YourCompanyName SomeTag #-- The directory path mentioned above should not contain a leading forward #-- slash (it needs to be a relative directory path). If the directory path #-- within the repository does not already exist then it appears to be automatically #-- created when you issue this command. #-- Check out the Repository modules file 'modules' with cvs checkout CVSROOT/modules cd CVSROOT #-- Text edit the 'modules' file to add a line like #-- yourModuleName repository/relative/path/to/code #-- Now, commit the changes to the modules file with cvs commit -m "Some Comment about what you have done" modules #-- Now, get rid of your working copy of the CVSROOT directory #-- (This can't be done when you're actually in that dir, obviously) cd .. cvs release -d CVSROOT Now you can checkout a working copy of the code with cvs checkout yourModuleName In other words you don't need to worry about the directory structure within the cvs repository. This assumes that you are working on the same machine as the cvs repository. See below if this is not the case. To set up the cvs server (to allow for cvs access across a network or across the internet), do the following, roughly: Add a line to /etc/inetd.conf refering to the pserver Restart inetd with kill -HUP Create a file /var/lib/CVSROOT/passwd with one line such as anonymous: This will need to be changed to stop any old person making changes to the Alexis code over the internet. Look at the actual file to see a better way to do this. Try connecting from a different computer with cvs -d :pserver:anonymous@ella-associates.org:/var/lib/cvs checkout geocities where geocities is a test module which matthew set up. http://ccvs.cvshome.org/fom//cache/124.html Here is a short spiel on how to setup a cvs pserver with xinetd Created two accounts for the cvs pserver (which is the client/server version of cvs, in other words the accounts to be used when accessing the 'local server' computer from across the network or across the internet.) The two accounts are -->> password hidden --<< The cvs server can be accessed with the following command line cvs -d :pserver:nicko@ella-associates.org:/var/lib/cvs login To create these accounts, I had to cut and paste an encrypted password from the /etc/shadow file (since we are using 'shadowed' password, otherwise it would be the /etc/passwd file). I used the 'adduser' program to generate the encypted password, even though it was not necessary to actually add a user to the system in order to use cvs. See the 'cederqvist' manual for more details. You could also try this to generate the password (from the cvs-home faq) -->> perl -e 'print crypt "password", "sa"' >>$CVSROOT/CVSROOT/passwd --<< All this appears to be working in Netbeans as well. By using the menu items 'versioning/mount version control/cvs' and then following the the steps of the wizard. The 'Connection Method' Box of the Wizard is the most tricky. Leave the connection method as 'password server' -->> Server Name = ella-associates.org User Name = nicko Repository = /var/lib/cvs port = 2401 (no need to change this) password = as supplied elsewhere --<< * March 9, Sunday not much * March 10, Monday not much. NetBeans OpenApi familiarisation * March 11, Tuesday xml and Netbeans research * March 12, 2003, Wednesday not much, fixed network card dongle connection looked at the chakirya server source code. * March 13, 2003, Thursday updated this file. A unusual Apache quirk: the default document for a directory, for example 'index.html', gets 'cached' by the web-server, so that even when you change the index.html file, or delete it, the changes are not apparent, unless the user specifically requests the document by name, e.g. www.ella-associates.org/index.html. Requests to www.ella-associates.org/ will yield the old document. I presume this is a feature rather than a bug. This is a strange problem. I even tried restarting the server with 'apachectl restart' but the old index.html file is still being served by the web-server. It just wont go away. When I put and index.html file in nhe 'test' subdirectory, the same thing happens but after it is deleted it eventually goes away. Could this be some odd client side problem with Internet Explorer refusing to not cache the document? Who knows. A slight 'gotcha' with restarting the apache webserver. You actually have to be in the /etc/init.d/ directory or the 'apachectl' script throws an error (cant find some file) This situation is especially odd considering that I have changed the index.html file for the document root directory /var/www/ before. * March 14, 2003, Friday A wild guess: could the problem above have something to do with Tomcat? Installed 'vim', an enhanced version of 'vi' The network card connection for this computer is slightly dodgy. Sometimes it is necessary to lightly press downwards on the network card connector in order to get it to work, or even to balance a not-too-heay object on the connector cord. The network card connector relies on a kind of 'pincer' action on the network card connection in order to work. But the gap between the two sides of the 'pincer' is now slightly wider than it should be, hence the problem. The Apache caching problem is incredibly infuriating. In MS Internet Explorer, in the menu, tools-->options [general tab, temporary internet files box] -->settings-->check for newer version of pages-->every visit By setting this option, the situation is improved slightly, but the old index.html is still being served Created a file 'ella-associates.org/docs/resources.html' which contains some url links to resources related to the Alexis project. Created some basic filter scripts in /var/www/utils/ to turn text into html In the /etc/rc files there are references both to apache and to apache2. This does not seem wise. How does the operating system know which daemon to start up at boot-time? Created an apache password file at /etc/apache/passwd/ella-members-passwords This is for allowing http access to the /var/www/internal/ directory. Created a password protected directory tree at http://www.ella-associates.org/internal/ The index.html file for http://www.ella-associates.org/ has finally actually been 'registered' by Apache. I really have no idea why. In other words the problem which I have been refering to, at length, about how the 'index.html' file which was served by the Apache Server was old, not up-to-date, and in some cases didn't even exist any more, that problem, has now mysteriously gone away. Created a set of filter scripts in the directory /var/www/utils/ which turn various types of text documents into html documents. Some of these filters are 'diary2html.sh', plaintext2html.sh, and linkdoc2html.sh. The filters also add links to the Google automatic translation engine for Spanish, French, Italian, German, and Portuguese. * March 15, 2003, Saturday Installed pretty good privacy, pgp on the Almetlla De Mar laptop. The outlook express plugin does not appear to be working but it doesn't really matter. I used the 'netword associates' pgp, version 6.5 something I think. And I got it from http;//www.pgpi.org/ Played tennis with Alex. Lost 6-3. Created a 'quick guide' to pgp for any other Ella Associates person who needs to install it * March 16, 2003, Sunday The problem refered to above about apache or MS I.E. not serving or displaying the most recent versions of pages, seems to intermittently re-occur. Tried to start the telnetd daemon from within inetd, by enabling the telnetd line in /etc/inetd.conf and doing a 'kill -HUP 1496' to restart inetd. The telnet daemon started and permitted a login but on successful login says 'cant execute /etc/ftponly' I wanted to start the telnetd for the purpose of debugging the DNS set-up. I probably don't need telnet to test dns anyway. The 'dig' tool is better. Am reading the DNS Howto which is at http://www.tldp.org and at http://langfeldt.net/DNS-HOWTO/ But, its not very helpful about how DNS works, and its written in poor english with plenty of mistakes. Nevertheless it is useful. There seems to be a reasonable manual which is installed with the 'BIND' package. This manual is written by the people who create and maintain BIND. It is available in pdf and html format. Also, perhaps refer to the document http://www.ella-associates.org/alexis-info/docs/dns-learning-materials.html * March 17, 2003, Monday ill, didn't work * March 18, 2003, Tuesday ill, didn't work * March 19, 2003, Wednesday Started the document /var/www/alexis-info/docs/the-ramble.txt This is supposed to be a simple explanation of Alexis ideas aimed at some-body like Marcus. James Info: to manage use 'telnet localhost 4555' and login with 'root' and 'jimi57' * March 20, 2003, Thursday Still sick * March 21, 2003, Friday Still sick. made email users 'mail' and 'webmaster' Made a summary of the Chakriya source code with the following command- -->> # (echo "

Chakriya Source Code

"; for f in *.java; do echo "

$f

"; cat $f | \ indent | /src-highlite/bin/source-highlight.exe --src-lang java --out-format html; done; \ echo "") > all.html --<< I wanted to then use this highlighted code (which is html) as input for the 'htmldoc' processor. The htmldoc program (GPL on unix) is able to convert html to Adobe pdf format. However, the htmldoc program does not recognise all html tags and for this reason the pdf output was 'scrambled' or 'garbled', that is, there were sections of the pdf text which contained overlapping and super-imposed words and also some of the source document words were missing. In addition the 'indent' program is designed to format or indent c code rather than Java code and this also caused the output to be less than good. * March 22, 2003, Saturday. Still ill. Investigating the Chakriya Source Code further. Added slightly to the document 'ChakDoc.txt' which is an attempt to document the Chakriya Server, but is still very rudimentary. Using the 'curl' tool to investigate the Chakriya Server from the command line. I am having difficulty actually working out the xml syntax or dtd for the requests. This syntax can be obtained from the 'output' windows of the Alexis Client Application It seems possible to communicate with the Chakriya Server with something like -->> curl -d "" \ www.ella-associates.org/alexis/alexisServer --<< This is obtaining a response from the server, sometimes and empty tag and sometimes a Stack Trace server error, if an illegal tag is introduced. * March 23, 2003, Sunday Not much * March 24, 2003, Monday Working on security breaches that mustafa (mu@ella-associates.org) has found. Mustafa appears to have been able to enter the 'local server' through the ftp server and appears to have been able to enter each of the guest accounts. He does not appear to have gained root access, but was able to upload files. Investigating Nicks problem of the javascript tree view component. At www.treeview.net there is a good example of a pure javascript treeview component that seems to work well, but there is no xml data-binding kind of capability, although it should be resonably straightforward to write something to do that. * March 25, 2003, Tuesday made a user 'webdeveloper/webdeveloper' who's home directory is /var/www/ This was to allow josephine to work on the 'ella-associates.org' site. However this is probably not the best or most secure way of doing this. The security problem seems to be that the ftp server is allowing anonymous ftp, and a script that I wrote, 'make-ftp-guest' had a bug in it that chrooted all guests to the /home directory instead of to the /home/[username] directory. This could be the reason that mustafa was able to put files in everybody's home directories. Prevented anonymous ftp access to the 'local server'. Also prevented access for the 'root' user. * March 26, 2003, Wednesday The 'enter/return' key on this laptop is becoming unresponsive (has improved since). Created a small email program set-up how-to for ella-associates.org which is at http://www.ella-associates.org/alexis-info/docs/ella-email-how-to.html * March 27, 2003, Thursday Created a set of backup scripts at /usr/local/bin such as 'backup-ella-web.sh' and 'backup-cvs-repository.sh'. These scripts backup the ella-associates.org website and the local-server cvs repository respectively. They also 'rotate' the previous backup copies. These scripts are scheduled in the /etc/crontab file to run automagically at certain intervals (approximately once every 2 days in the early morning) My documentation pages and alexis information have been moved to http://www.ella-associates.org/alexis-info/ * March 28, 2003, Friday Expanded the document 'javascript-treeview.txt'. Continuing to investigate the use of javascript tree-view components in a web-page. * March 29, 2003, Saturday More Java/Script Tree View Component Research. Going to a meeting in Barcelona with Nick tomorrow, 5pm * March 30, 2003, Sunday Went to a meeting in Barcelona. Did not work Some of the things suggested for me to do were: Implement an IDE Auto-updating feature for Alexis Implement a schema validation for Nick XML tag editor. Implement Colors for the tag editor. Implement some chat system or discussion forum for Alexis. Investigate XML Databases. Send the pgp how-to link to everybody in the 'team'. Send the XML explanation: Done link to everybody. Set up 'grapevine.ella-associates.org' and 'language.ella-associates.org' (possibly not urgent because they won't be used immediately.) Do the necessary massaging of the Lexicographical test data so that it can be inserted into Alexis. Investigate a Tomcat 'glitch' which appears to stop large numbers of accesses in very short periods of time. This behavior was reported by Nick. * March 31, 2003, Monday Returned from Barcelona. Did not work * April 1, 2003, Tuesday Not a great deal. Extended 'The Ramble'. Revised some docs. Created some tables of contents. * April 2, 2003, Wednesday Revised Documentation. Began to look at Tagged Editor * April 3, 2003, Thursday Looked at available xml editors. Reviewed some web collaboration software. * April 4, 2003, Friday More Collaboration software. See the review at 'resources.html' Can a working discussion forum for Ella be easily set up? yes (YaBB). Would it be worthwhile? I dont know. It may be 'just another thing' that people have to check. It may be better to wait until it is implemented within Alexis. * April 5, 2003. Saturday Questions: what is IRC how does it work? what are the implications of using GPL software in a commercial venture? Some parts of the code will have to be given away. To do: The Tag Editor Familiarization. Get j-ftp. Wash up. * April 6, 2003, Sunday Sat in the sun. fixed another Robert Hotmail problem (the same problem as befor) to do: still tag-editor familiarization. Off-site backups. * April 7, 2003, Monday Day off * April 8, 2003, Tuesday Another day off. Had a business meeting with Nick in 'pica pica'. * April 9, 2003, Wednesday Emailed Vanessa, made vim 'filter' aliases. For example, in order to turn this file you are reading (assuming that you are reading the 'plain text' version, and that you are reading it in a text editor called 'vim'), in order to turn this file into HTML, all you have to do is type ',3' (that is: ) from within the vim editor. You don't actually type those single-quote characters ('), they are just there to separate the character sequence from the rest of this text. This character sequence runs the script 'diary2html.sh' on the current file. If you type ':map' you can see what other character sequences are available. Read interview with Bill Joy at 'linux-mag.com'. Notes for making a backup script for the Alexis Source code and Data The script can use a cvs 'update' command to check out recent changes to the alexis source tree from the (putative) server in the U.S. to the 'local server'. I say putative because this U.S. server has not yet been established but possible will be over the next few days. To test this functionality, I can run the script from the cygwin shell, or another, from the 'ametlla' laptop. This method of using cvs to achieve the backup allows us to do 'incremental' backups without having to code that functionality ourselves. This incrementality greatly reduces the amount of data that has to be transfered over the Internet. A possible disadvantage of this, is that the versioning information that is contained in the CVS repository, will be lost. Also, the MySql database needs to be backed up. This will be hopefully achieved by using some MySql incremental backup facility. Also it is envisioned that the Alexis Client could be easily modified (or extended) to act as a backup agent considering that it will already incorporate functionality to allow for the caching of data and for the incremental 'refreshing' of that data. Also the 'log' table of the MySql database, which is the table which stores versioning and change information for the Dictionary data used by Alexis, will need to be backup separately because the Alexis client will not be able back this up. also to do: create a symbolic link from ../alexis/docs/ to /var/www/internal/alexis-docs/ or something similar. Or else create an Apache 'alias' for this. Creating an alias may be difficult because the 'internal' directory already exists, and because I am not sure if authorisation restrictions will apply to the aliased directory or to the symbolic link. * April 10, 2003, Thursday Little things: I should modify the 'filter' scripts so that they recognise the 'query string', that is anything after the ? character as part of the url. At the moment this is especially a problem for the 'mailto:' hyperlinks Little things: Create a general 'ella' account for the 'internal' dir Created a CVS module name 'alexis-docs' which points to the ella/alexis/documentation directory withing the CVS repository. This module name is going to be used to publish the documentation on the apache web server at the url http://wwww.ella-associates.org/internal/alexis-docs/ .This can be achieved by using the cvs checkout command when in the 'internal' dir in the following way cvs checkout alexis-docs or if the /var/www/internal/alexis-docs dir already exists use the 'update' command. Imported the /var/www/utils/ and /var/www/alexis-info/docs/ directories into the CVS repository. Set up the CVS repository with a definitive alexis code tree (imported the code into the repository). This tree is now going to be used for all development, which means that all alexis developers (i.e Nick Reddel) must use CVS when developing. Thankyou. [note: in reality cvs was never really used, neither for code development nor for the documentation] This import failed because the binary files were treated as text by the CVS program. It is necessary to explicitly inform CVS which types of files are binary. The binary files extensions which we need to inform CVS about are as follows gif jar png zargo zip exe jpg jpeg act ACT psd PSD pdf rar eps ai doc xls ppt fla swf But what about '.class' files, for example? [note: I would like to congratulate myself for recording this detailed information since I forgot to get the appropriate cvs configuration file from the Barcelona Debian Server] The line below apparently tells CVS that 'exe' files are binary, but it may be necessary to put multiple expansions in a file called 'cvswrappers' (yep). This info is from the rather 'garden-pathish' cederqvist manual for CVS, which is the main manual. -->> cvs import -I ! -W "*.exe -k 'b'" first-dir vendortag reltag --<< I placed the necessary lines in the 'cvswrappers' file, which is at /var/lib/cvs/CVSROOT/cvswrappers (but this file should be edited by doing a cvs checkout CVSROOT/cvswrappers command, editing the file and then doing a 'commit'. This appears to have worked and the source tree appears to now be happy lying in the repository. Nick's original ftp'd source tree is at /home/matthew/alexisSource and a 'test' checkout of the code is at /home/matthew/alexis. In order to update the web publishing of Nick's documentation you do the following cd /var/www/internal; cvs checkout alexis-docs This appears to do the trick even if the directory 'alexis-docs' already exists. Also should have a look at a thing called webDAV which may be a type of cvs for non programmers. (No it is a protocol based on http which allows the collaborative editing of documents) Crazy ideas: create an Alexis and also Ella faq-o-matic (http://faqomatic.sourceforge.net ) and an Ella and Alexis YaBB forums It would also be useful to have 'discussion' pages where people could contribute by clicking on a hyperlink. A windows NT line to get a password protected page using wget -->> D:\mjb\utils>wget --http-user matthew --http-passwd bacibaci www.ella-associates .org/internal/ella-internal-docs.tar.gz --<< A bash line to backup the /var/www/internal directory excluding the large 'all-chak' documents and Nick's documentation directory which is also quite large. -->> cd /var/www/internal/; rm -f ella-internal-docs.tar.gz; tar -czvf ella-internal-docs.tar.gz [^a]* | less --<< A bash line to backup the docs directory excluding the large 'netbeans-userguide' files. The less pipe is not really necessary but allows the user to see what files are being 'tar-balled'. This relies on the fact that the only files in the directory beginning with 'n' are the netbeans user guides. cd /var/www/alexis-info/docs; rm -f ella-web-docs.tar.gz; tar -czvf ella-web-docs.tar.gz [^n]* | less Started a script which will add comments or other types of information to a text or html file. * 11 April 2003, Friday Need to change my plane ticket. Overcast day. Backup script is still to do. Created a prototype document commenting script, which allows a reader of a document to add comments or other text to it. It is located at /usr/lib/cgi-bin/add-comment. Also I created a text2html script (plaintext2html-forum.sh) which is designed to be used in conjunction with this cgi script. The filter script creates the input form and the necessary links to the cgi script. These scripts are to be rewritten in Java Servlets or JSP or PHP and are to use a MySql storage system and to support threading. I have set up the filter scripts so that the main repository is at /var/www/utils/ but most of them are 'symbolically' linked to the /usr/local/bin/ directory. This puts them on the 'path' and allows them to be used from any directory. Created an ftp guest user 'nick' Added a line in the /etc/vim/vimrc file to map the ,1 key sequence to running the forum generating script. map ,5 :exec "!plaintext2html-forum.sh " . expand("%:p") . " > " . expand("%:p:r") . ".html" Essentially finished the prototype version of a 'forum' script which allows a user to add a comment to a text file. This now needs to be translated into JSP, given a MySql database and made to support threading. * 12 April 2003, Saturday Didn't really work. Visitors (kerry lewis) arrived. Thought about the potential of text messages. When to la quadra restaurant * 13 April 2003, Sunday Read Nicks 'project overview' document. Revised the main documentation page. I can send an email with 'telnet localhost 25' on the local server * 14 April 2003, Monday Struggled with 'netcat' and sending an email to the 'local server' james mail server using a bash script. A detailed description of this process is available in the file 'smtp-torture.txt' * 15 April, Tuesday Note that the output of 'linkdoc2html-forum.sh' does not display in Netscape 4.61 On Win NT, 2000 or whatever you would like to call it, the following command line from within the 'sh' shell, when run from d:\mjb\ella-backup-site\ will create a tarred and gzipped backup of the utils subdirectory. The backup is given a datetag for the current day. If more than one backup per day is desired this would have to be altered by adding an Hour tag. Run 'date --help' to see what format tags are available for 'date'. Note on WinNT there is already a 'date' program thats why I renamed the unix one to ddate. DIDO# datetag=$(ddate "+%d%B%Y"); tar -cvf utils.$datetag.tar utils/*; gzip utils.$datetag.tar created a backup script to get the stuff I have been doing off the local server and onto the Amettlla laptop. It is at d:\mjb\ella-site-backup\backup-ella-site. Apologies for the name. It also archives old backup copies. It needs a bash shell to run. I attempted to schedule it in Windows 2000 with at 19:45 /every:M,T,W,Th,F,S,Su sh backup-ella-site Apparently 'at' is the MS Windows 2000/NT scheduling command (did they hire someone with a unix background?). The only real doubt I have is 1. Is the NT scheduling service running? and 2. will the at command pickup the 'parameter' to the 'sh' command. Did not work. But the follwoing did work at 05:05 /every:M,T,W,Th,F,S,Su cmd /c d:\mjb\ella-site-backup\start-ella-backup.bat where 'start-ella-backup.bat' contains -->> d: cd \mjb\ella-site-backup echo "started ella backup" >> start-backup.txt sh backup-ella-site --<< So the backup is scheduled for 5.05 am every morning. These techniques should be easily applicable to the Alexis Repository Backup. The only problem with this is that the laptop in Ametlla may not be switched on a 5.05 am. It probably wont. Added warcontent and jarcontent to the 'cvswrappers' file at /var/lib/cvs/CVSROOT/cvswrappers Started a backup script for the alexis repository at d:/mjb/ella-site-backup/backup-alexis-repository or something similar. Started a php page which will send an email. Actually I think this should go in the /var/www/index.htm which would become a php file so that a 'confirmation message' could be displayed to the user. It may be worthwhile setting up 'sendmail' instead of 'bashmail'. Bashmail is a simple 'bash' unix script which I wrote which sends an email through the SMTP server on the localhost. * April 16, Wednesday Improved the bashmail.sh script so that the email is sent quickly (put a 'sleep' line in the mail generating script). Some quick php syntax research. Finished the 'ella-associates.org' e-learning contact form * April 17, thursday Did not work * April 18, Friday Fixed the 'checkhotmail' java script. Began to configure the U.S. Server. * April 19, Saturday Stuff to do: repository backup script, on US server install java, tomcat, transfer files, change apache document root. Roberts script. * April 20, Sunday Some investigation of how to validate against a schema using xerces. The answer, may be, is Document.normalizeDocument and the page explaining it is at ` apache.org/xml-xerces/java/build2.3/docs/faq-dom.html#faq-6 but really this is not what we want at the moment. We want to know what are all the valid elements which could possibly be inserted in the current context. Apparently to do this we should use something like PSIV. Look at the faq in the Xerces documentation for how to do this. * April 21, 2003, Monday Didn't work. Did some house cleaning prior to the arrival of Garry. It was very sunny. Went to dinner at la cuadra. Drank whiskey and wooden-barrel wine. * April 22, 2003, Tuesday Still have to link the 'Tomcat' java application server with Apache. The documents for Tomcat dont really look that good. * April 23 to 26, 2003 Haven't done anything. Distracted * April 27, 2003, Sunday Finished the 'checkhotmail' web script. This is at http://www.ella-associates.org/cgi-bin/check-hotmail The user can check if Hotmail Email Addresses are valid or not. * April 28, 2003, Monday Worked on a grapevine band editing capability. Found some interesting sed resources Wrote an HTMLentitizer script to transform special characters into HTML entities but then discovered this has already been done (better) at sed.sourceforge.net [Actually I strongly suspected this would be the case before I started writing the script, but I was unable to find the script. As it turns out, it would have been virtually impossible to find the relevant sed script using normal Google searches. I only came across by accident when investigating other capabilities of sed] Although the writing of this sed HTMLentitizer script may be seen as a bit of a waste of time, I did learn a lot about transforming 'control characters' in the process of writing this. * April 29, 2003, Tuesday Not a great deal. Obtained and integrated a style-sheet for the alexis documentation * April 30, 2003, Wednesday Grappling with US server problems. Attempting migration to the US server. * May 1, 2003, Thursday The problems with the US server: Intermittently it is not possible to log on. Also, once logged on the connection is 'aborted' quite frequently. This may be a problem with the way the the SSH server is configured. This problem is currently causing the US server to be unusable. The connection 'freezes' after a minute or less. Also the 'less' pager is displaying hyphens as a's with hats The solution to this problem may be the '-7' or --ascii switch to man According to nick the solution to this is to set the character set in 'putty' to utf-8. for the US server I may try killall -HUP sshd in order to restart it. also will try telnet to see if that helps. * May 2, 2003, Friday * May 3, 2003, Saturday Noticed some very odd stuff in the '/root' directory. Don't know what it is. A little bit of 'proftpd' familiarization. Proftpd is probably a simpler ftp daemon or server to use than wu-ftpd (which is what the Debian Linux Server is Barcelona uses). In order to 'chroot' a user into his or her home directory, use the 'DefaultRoot' directive in the proftpd.conf file. These directives a very similar to Apache style directives. The odd stuff in the /root directory on the Barcelona Server was just the result of an in cautious right-click, rather than some dire hacking intrusion. Cleaned this up. * May 4, 2003, Sunday Sat at the corner table at the La Cuadra restaurant as the sun went down, and was joined by various people, including Sara and Jordi and Daniel. Drank some coffee and listened to some songs by Daniel. Grappled with the U.S. Server configuration. AT the moment I am looking at how to set up the 'James' mail server. This is possibly very simple but I tend to need to understand everything before I start. Also I looked at sendmail but was repulsed by its arcane and ridiculous configuration. I am happy to ditch it and start again with James (although sendmail is already installed on the Norte-Americana Server). Things still to do: copy all of vanessa's stuff over to the US server set up 'webmin', set-up the 'cgi-bin' directory, notify Josephine of stuff, notify everybody else, fix the DNS rollo, :w fix this anoying 'vim' wordwrap thingo, go for a swim, install james, fix the character-set problem with putty and the US server. * May 5, 2003, Monday Another James day. To set system wide environment variables, put them in /etc/profile or the 'init' script. James was eventually persuaded to start and run by stopping the existing Mail Server configuration which existed on the United States Red Hat LInux server. This system involved an application called 'sendmail' and its various related programs. As I was able to understand it, there was a master program or 'daemon' which controlled and 'spawned' the other applications. It was possible to stop this master daemon, which was identifiable in the process list as 'master', by typing the command 'postfix stop' Also it was necessary to stop the default pop3 server as well as the SSL enabled pop3 server. These programs were controlled by the 'xinetd' daemon which is sometimes referred to as a 'super-server'. By finding the correct configuration file and setting 'disabled=true' in the ipop3.conf sub- configuration file, it was possible to stop these demons from causing any further trouble. If these defaults daemons are already running it is not possible to start the James Mail Server. Sadly James gives you a large and not overly helpful 'stack-trace' instead of a simple 'port taken' message. Although in the large stack trace there is a 'port taken' message somewhere if you look. * may 6, 2003, Tuesday stuff. Worked on the problem of connecting the Apache Web Server to the Tomcat Application Server. Both of these applications are developed under the Apache Software licence at http://www.apache.org The process of joining the 2 applications involves using the mod_jk module which is refered to as an 'adapter' in the documentation. The on-line internet based documentation for Tomcat is somewhat difficult to follow, probably because of the rapidity with which the Tomcat application has been developed in the last few years. One example is that the documentation for version 4 of Tomcat refers to 'startup' scripts which are called 'startup.sh' and 'startup.bat' but these scripts do not appear to actually exist anymore. Some things which probably need to be done, still, are: To test Nick Reddel's Java based XML editor; To look at the Amaya application which is something that has been developed under the auspices of the w3c and which apparently has editing capabilities as well as being an Internet Browser. As Nick Reddel has put it, this is possibly the application which is most similar to 'Alexis' at the current time. (There is a commercial product called XmlMind which is also slightly similar). N. Reddel would like FTP accounts for the following people Nick, Matthew, robert, mustafa, marcus, jo, vanessa, monkey-rhythm, poseidonia (all with ~ - type public html dirs) Other things: How can web-applications in the same instance of Tomcat send data and information to one another josephine/josephine monkey-rhythm/m0nkey * may 7, 2003, Wednesday The apache web-server appears to need 'execute' permission on the directory for the guest users. That is, the directory /home/username seems to need to have its 'execute bit' set for 'others' in order for apache to be able to display the contained web-pages. The /home/username/public_html directory needs both the 'read' and the 'execute' bits set (I presume for 'other' since apache is running as the user nobody) It would probably be a good idea to put a skeleton 'public_html' directory in the 'useradd' command skeleton directory, with the correct permissions so that as soon as the new user is added the public_html directory will be available The checkhotmail script is not working properly, possibly because of file permissions for the 'temporary' file [The problem is now fixed- it had to do with the location of the 'java' executable and the usual Unix permissions problems]. Or permissions for the directory. The 'filter scripts' (in /var/www/utils/) are not all working properly because of some problem with sed. plaintext2html.sh works but not plaintext2html-forum.sh [PROBLEM NOW FIXED, it had to do with the '{,4}' range regular expression syntax] There is a problem at the moment that the server is rejecting requests to ports other than the very common ones (80 etc). For example it is possible to access the 'webmin' server which is called miniserv.pl using the comand lynx http://localhost:10000/ but if the IP address is used from outside of the American Server, it rejects the request with a message such as 'connection refused'. Otherwise Webmin seems to be nicely installed and does what its supposed to do. Currently it uses the system root password, which will be changed prontissimo Created various user accounts. There was a problem with the 'James' server dying. This was caused by it not being properly started using 'runlevels'. Instead I was simply starting it in a normal 'putty' session. It is necessary to put scripts in /etc/rc.d/ ... which get executed when the run-level is changed by the operating system or by the user. The operating system changes the runlevel when boot the system and when shutting it down, but the runlevel can also be changed manually by a root user. This means that the process (such as a James mail server) is not owned by whoever happened to start the process, but it owned by the runlevel, in a sense. If this is not done, then when the user who owns the process logs out, the process is killed although I am not sure if this has to be the case. * 8 May 2003, Thursday The following problems with the American Server remain: Something is killing the James Mail Server daemon, The Server is rejecting requests on most ports except the common ones, the Java Servlets are returning 'no data' messages when they are accessed The hosts.allow and the hosts.deny need to be edited to allow new services. By commenting out the line ALL:ALL in hosts.deny file it should be possible to allow complete access. However, nevertheless the American server is still rejecting requests to http://63.105.73.195:10000/ So there is some additional problem apart from the hosts.deny file Also the 'portsentry' program does not seem to be the problem because even when it is killed, the connection to webmin as above is still refused PROBLEM IS NOW FIXED: caused by iptables. The above problem was caused by a thing called 'iptables' which is a piece of software which is part of the linux kernel. Iptables is something that is often seen in connection with 'firewalls' and are a method of preventing 'denial of service' attacts against a computer as well as other hacking attempts. Nick solved this iptables problem by 'zapping' the iptables configuration. In other words the configuration has not been properly re-organised, it has simply been disabled. Here is the email from Nick Reddel about how the problem was solved --<< The prob was that james had the wrong java_home (had to put it directly in phoenix.sh, cos environment vars don´t stick when running from a non'shell environment) and then tomcat wouldn´t start. changed k20tomcat4 to s20tomcat4 in /etc/rc3.d (this may be an issue for a later full reboot)(probably not) ummm...and i seem to have zapped the boot logger (which was logging to boot.log)...cos i deleted boot.log to clear it, then touched it, but it don't like me. alas but all is well on the system front yes, now I'll read the boot.log manual n --<< Tomcat is still not 'joined' to the Apache Web Server but the solution to the problem my simply be that Tomcat needs to be restarted, which is something which I never did after installing the 'mod_jk' module. * may 10-14 FA * may 15,2003 Fixed the checkhotmail cgi script on the US server. Set the US Redhat Server to Barcelona Time, since the majority of users are in Barcelona. Fixed the 'add-comment' scripts. Had a short nightmare trying to start apache with the script in /etc/rc.d/init.d/httpd Dont do this because it tells you that you are using 1.3 directives and it wont start the server. Use /usr/sbin/apachectl instead I established a reasonable 'error-document' for the apache server on the American Host. These error documents now point back to the main index page for Ella Associates, that is 'resources.html'. * may 16, 2003 The 'public_html' directories for each of the users home directories need to be 'writable' by the 'other' user. In other words they need to have -->> chmod o+w public_html --<< done to them, if any comment script is to work in those directories. I stopped access to the 'guest' ftp accounts on the Barcelona Debian Linux Server, in order to stop people from accidentally using that Server instead of the new American RedHat Linux Server. I fixed the script which sends and email from the 'e-learning' home page (the 'netcat' utility needed to be installed on the American Server). I updated the 'resources.html' file to provide links to the Ella Associates contributors home web-pages on the American Server, as well as links to the 'grape-vine' page. * may 17, 2003, saturday Some webdav familiarization. Tomcat investigation. The Tomcat examples are not installed correctly. But the Servlet examples are working. * may 18, 2003, Sunday Some Tomcat reading. * may 19, 2003, Monday Still on Tomcat. Am going to copy over the '.bashrc' file for the root user from the Barcelona Debian Server. Also, I should add 'aliases' for all the important 'log' files, such as Apache, Tomcat, James, etc etc. This will mean that I wont have to remember the directory locations for all the log files. I could also do this for the 'home' directories for the various applications. Yep. I have created some of the aliases mentioned above and put them in the file /etc/bashrc So they should be available to all users, rather than just the root user, as would have been the case if I had put them in /root/.bashrc In order to see all the available aliases type 'alias' The aliases are in the general form -->> applicationName.functionName for example tomcat.log will display the tomcat log file --<< The lynx help files on the American Server need to be fixed. Tomcat is successfully serving JSP files to the localhost as well. For the Tomcat Apache problem, the permission of the mod_jk.so file may need to be changed. NO, probably not the problem. Or the the actual mod_jk.so binary need to be changed to an SSL enabled version (since Apache is running with SSL). I have tried this, but didn't work Or the 'autoconf' mode for tomcat should be used. Or the mod_jk.conf file is mounting stuff incorrectly. Or the tomcat 'server.xml' file may need to be changed in some way Probably not since it already has the 'ajp13' connector defined The only 'log' file which appears to display any kind of error message is the apache 'error_log' which displays a very cryptic error message When I make a request to a Servlet or JSP using the mod_jk 'mapping' into the normal Apache 'document space', the apache error log records the following -->> [Mon May 19 14:43:18 2003] [notice] child pid 14393 exit signal Segmentation fault (11) [Mon May 19 14:43:19 2003] [notice] child pid 16725 exit signal Segmentation fault (11) [Mon May 19 14:43:19 2003] [notice] child pid 14403 exit signal Segmentation fault (11) --<< The mod_jk log file does not appear to contain any error messages. The URL below is a pretty helpful article about this mod_jk/ apache problem which I am having. It suggests that a connector needs to be added to the Tomcat server.xml file. But that connector is already there. http://www.experts-exchange.com/Web/Web_Servers/Apache/Q_20411439.html#7553532 This article also at some point recommends the solution which was eventually used at Ella Associates: recompiling the mod_jk.so file * May 20, 2003 * May 21, 2003, Wednesday Pottering * May 22, 2003, Thursday A problem with the American WebServer. Pings to www.ella-associates.org work. Telnet www.ella-associates.org 80 This may be a problem with the Telefonica DNS Server. The Apache Web Server located on the American RedHat Server appears to be functioning normally and requests that are made from within the Server itself, using a 'Putty' command session, using the command -->> lynx localhost --<< This command displays a textual version of the Default Web Page for Ella Associates. Also, when the Default Web Page is requested using the I.P. Address of the American Redhat Server, the correct web page is displayed. AT 4:00 PM the Web Server Problem is still occuring. At the moment there is some 'default' setting for the VIM text editor on the American Server which auto-matically wraps or folds the text lines once they exceed a certain number of characters, which number is approximately about 80. The difficulty involving the representation of certain special characters on the American Server, through the medium of a 'Putty' login or command session, currently continues. In the pueblo of Ametlla de Mar, a festival of music is being organised to occur on the 15th of August 2003. A number of different musical groups will be playing. To provide information about this event a website will be established which may contain information about the Bands, including examples of their music as well as other details about the event. I will be contributing towards this web-site. The Hub is a newsletter orientated towards the locality of 'Baix Ebre' which, I believe, is the region in which the town of Ametlla lies. The Hub appears to be written mainly for an English-speaking audience, although it is possible that it is also translated into Spanish or Catalan. Currently it seems that the newsletter is to be published only on the Internet after a period in which it was also distributed in a 'hard-copy' form. This periodical probably should be notified about the Festival of Music which will occur on the 15th of August, 2003. The web-site which is located at http://www.ella-associates.org and http://www.ella-associates.org/alexis-info/docs/resources.html is currently hosted in the United States of America. It would seem useful to mirror this site onto the Debian Linux Note: N.Reddel has solved the Tomcat/ Apache integration problem by recompiling the 'mod_jk.so' file on the American Server. Once the file has been manually compiled it is 4 times larger than the file which is provided on the www.apache.org site, in the release directory of the 'mod_jk' module. It is still required to set up 'Bugzilla' and 'IssueZilla' and 'Cocoon' as well as to move the 'crontab' file over from the Barcelona Server to the American Server in order to enable the periodic backups of important directories and files. The correct format for a 'multi-lingual' web page is as follows pagename.pagetype.languagecode For example, a page in Spanish should be called index.html.es Microsoft Internet Explorer also seems to understand a format which is exemplified by the resource name index.es.html but from my somewhat limited knowledge this resource naming format is not in accordance with the 'standard', whatever that may actually be. This standard is probably an RFC 1am in the Morning: the DNS problem (assuming that it is a DNS problem) is still occuring. From an Ametlla De Mar laptop computer which is running Microsoft Windows 2000, doing a nslookup www.ella-associates.org returns the following information -->> D:\mjb\ella-site-backup\utils>nslookup www.ella-associates.org DNS request timed out. timeout was 2 seconds. Can't find server name for address 80.58.0.33: Timed out Server: 97.Red-80-58-32.pooles.rima-tde.net Address: 80.58.32.97 Name: ella-associates.org Address: 63.105.73.195 Aliases: www.ella-associates.org --<< From this result, we can possibly deduce that once of the Spanish 'Telefonica' DNS Servers is 'down' or not functioning at the moment, unless it is a configuration problem on the Barcelona Debian Linux Server. It shold be remembered that at the moment the Barcelona located Debian Linux Server is the 'authoritative' Domain Name Server for the Domain ella-associates.org. To simplify, when requests are currently made (1am today) to an Internet Address such as http://www.ella-associates.org/alexis-info/ an error message is displayed by the Web Browser such as 'the document contains no data'. This error message, from my experience, is generally returned by some types of Browser when there does not exist any Serving Application listening on the port required. For example, if there is no Web Server application 'listening' on the port 80 when a browser makes a request for an Internet Resource then the browser (notably Netscape), may return the message 'the document contains no data'. On the other hand, if a request is made to a URL such as http://ella-associates.org/alexis-info/ then the correct web page is returned. From my limited and rather superficial knowledge of the DNS system I would guess that this problem is caused by a configuration problem on one of our Linux Servers, rather than by a problem with the Telefonica DNs Server. Please note that 'Telefonica' is the main Spanish PSTN Service provider in Spain, equivalent to digamos, Telecom in Australia. In the opinion of N.Reddel (who seems to know significantly more than myself about these sorts of subjects), this IS a Telefonica problem This problem is also 'killing' the comment scripts on the American Linux Server because the HTML Form submit button appears to point to 'www.ella-associates.org' to some subdirectory of this domain name. * May 23, 2003, Friday The problem with the DNS seems to have fixed itself. For example, when a request is now made to a resource such as http://www.ella-associates.org/alexis-info/ The correct page is returned instead of the 'document contains no data' message as before. However, when doing an 'nslookup' from an Ametlla de Mar laptop as before, the same sort of results appear to be returned which possibly indicates that the Telefonica Server is still no available. For example, the following is an extraction from a console session on a Windows 2000 computer. -->> D:\mjb\ella-site-backup\utils>nslookup www.ella-associates.org DNS request timed out. timeout was 2 seconds. Can't find server name for address 80.58.0.33: Timed out Server: 97.Red-80-58-32.pooles.rima-tde.net Address: 80.58.32.97 Name: ella-associates.org Address: 63.105.73.195 Aliases: www.ella-associates.org --<< The 'linkdoc2html-forum.sh' script does not currently write a link to the Alexis Documentation home page. This was because originally the Documentation Home Page was a link document and for this reason did not need a link to itself. Also the scripts don't handle resource names such as index.txt.en very well. * May 24, 2003, Saturday Installed the Cocoon document server and renderer on the Ella-Associates Server. And now for 'bugzilla' http://www.bugzilla.org/ * May 25, 2003, Sunday Trying to install Bugzilla. This is not entirely straight forward since a large number of perl module dependencies are necessary. administrator account for bugzilla: user: matthew@ella-associates.org The database user for bugzilla is name bugs Bugzilla is mainly working but cant send email notifications of Bugs because it requires Sendmail to be properly configured. However the James mail server is being used instead of sendmail although it is possible that the two might be able to co-exist Copy a file called 'slide.def' from nicks E drive. Grant mysql permission only from the Ametlla IP address. Look at what is currently on the Barcelona Debian Server. Installing the FAQ-o-Matic Temporary Password There is a problem here in that in order to properly install the faq-o-matic it is necessary to have a 'mailer' properly installed on the host machine (something like sendmail for instance) which accepts the -s subject switch. However, our mailer on the US Server is James which does not seem to have a command line mailing interface. It may be possible to get around this either by directly editing the faqomatic configuration files, or by using my dodge 'bashmail' script and modifying it to accept the -s switch. I do not know if this 'hack' will work. In general the installation process seems unnecessarily complicated. Did the initial installation process for the apache 'slide' webdav server. I still need to enable the JDBC data store facility and to properly edit the configuration files * May 26, 2003, Monday Doing a little bit of pottering around with the Ametlla De Mar music festival web-site. This should probably be primarily in Spanish. Still having a serious problem with 'character sets' on the American Server For example when I type 'a acute', in the VIM editor, this is not stored as 'a acute' but as something like 'Capital A tilde, Ordenal O [like a degree sign]'. I still dont know if this is the 'putty' application or the character set used on the American Server. This is adversely affecting my ability to create a spanish version of the L'Ametlla de Mar Music Festival web pages, since I am unable to get the accented characters to appear decently. Trouble shooting ideas for the American Server character set problem: I could write the file on my local Microsoft Windows 2000 laptop and ftp the file to the American Server Interestingly, the 'vea esta pagina en blah blah' link which occurs at the top of the generated HTML pages, occurs correctly with accented characters, which is because I believe that I cut and paste this text from a Google Automatic Translation page. I could temporarly enable telnet, then telnet into the american server and create a file with accented characters, if these are not 'mangled' then the prob is probably putty, I suppose * May 27, 2003, Tuesday Looking at slide JDBC stores MySQL Data files for slide are at c:\mysql\data\slide\ On Nicks laptop. The slide.def file was at e:\slide\server\conf\slide\slide.def According to Nick, the process in getting the slide server to use a JDBC store is just; copy the mysql tables from the location mentioned immediately above to the American Server. Then change the ownership of the files and permissions to whoever needs them, this step I am not entirely clear about. Then put the slide.def file in its correct location on the american server and voila! con suerte * Wed Nope * May 29, 2003, Thursday Understanding Cocoon, and trying to fix slide. Understood sitemaps and generators and serializers. The Cocoon samples and docs are very good. * May 30, 2003, Friday A question about getting slide to work with tomcat and mysql http://faq.globalvision.com.au/fom-serve/cache/88.html The above URL contains very good step by step instructions for setting up slide with mysql, tomcat and creating a web folder to see the slide directory. The Apache slide documentation is generally fairly poor and tends to assume that you are a developer actually working on the code. Or something When Tomcat is stopped and started it will often create a directory called 'contentstore' in the current directory, that is, the directory from which tomcat is being started and stopped. This is caused by the 'slide' webapplication which uses that directory to store webdav content. Slide then uses this newly created content store directory to store all uploaded files, etc, which is bad Sometimes, in addition, Tomcat needs to be started twice in order for the 'slide' application to be accessible. These problems should be solvable by editing the files /var/tomcat4/webapps/slide/Domain.xml and /var/tomcat4/webapps/slide/WEB-INF/site.xml These are the only two configuration files which affect the slide web application. The configuration files in the 'slide' installation directory appear to have no effect, logically enough The following error generated when stopping tomcat (with say, the alias tomcat.stop) can just mean that tomcat is not currently running. -->> Catalina.stop: java.net.ConnectException: Connection refused java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:305) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:171) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:158) at java.net.Socket.connect(Socket.java:426) at java.net.Socket.connect(Socket.java:376) at java.net.Socket.(Socket.java:291) at java.net.Socket.(Socket.java:119) at org.apache.catalina.startup.Catalina.stop(Unknown Source) at org.apache.catalina.startup.Catalina.execute(Unknown Source) at org.apache.catalina.startup.Catalina.process(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at org.apache.catalina.startup.Bootstrap.main(Unknown Source) --<< The contentstore directory appears to be created the first time that the slide default page, or url is accessed, by only the first time after tomcat has been started. There is probably a bug in Explorer 5.0 and web folders, it doesn't appear to 'authenticate'. At the bottom of the Domain.xml file is a line wich allows guest privileges this should be uncommented until authentication is sorted out. Files which are copied into the contentstore/files directory after slide has started, do not seem to appear in the directory listings. Files which are already in the contentstore/files dir when slide starts, are ALSO not 'seen' or recognised. Dont know why * may 31, 2003, saturday Washed clothes, tidied house. 7:30 pm, a little work The main configuration file for jakarta-slide running as a Tomcat webapplication is /var/tomcat4/webapps/slide/Domain.xml the other is /var/tomcat4/webapps/slide/WEB-INF/web.xml The Domain.xml file seems to contains a sort of 'amalgam' of three configuration files which are used when slide is running as a 'standalone' application. +++++Things to try with the Jakarta Slide Server. I am currently still having problems with the material (ie files) in the web-folder 'disappearing' when tomcat is stopped and started. remove the 'slide-webdav' webapplication which I installed in a moment of ignorance and blind hope NOPE, doesn't seem to have improved anything Do a complete re-install of slide, making sure to preserve Nicks 'slide.def' file. There doesn't seem much point to this because I haven't modified many files and have been careful to preserve the default configuration files Set the 'resetsomething' parameter to true in the Domain.xml file then restart tomcat. Switch to a JDBC store using nicks slide.def file and inserting the relavent bit in Domain.xml Also examine the 'permissions' section of the 'Domain.xml' file and remove or comment out the bits which will confict with the permissions or ACL information in the JDBC database store. By doing this it should be possible to determine if slide is actually getting file and permission information from the Database or not. This process of going one step further in using JDBC stores may seem to be unnecessarily complicating things but I have sort of run out of ideas for the 'simple' file store. Also, if permission information IS being drawn from the database, then try emptying the 'objects' table, * 1 june 2003, sunday 2:15pm Some work I need to get a mysql client for windows so that I will be able to check that the slide database is accessible Or the alternative is to 'putty' in to the Barca Server and check from there. The requests to the Mysql server on the United States Server are causing the 'iptables' program to block all requests to the server. If this occurs the solution is to putty into the Barcelona Debian Linux Server and then use the 'ssh' program to get a remote console on the american server. You will need to know the root password for the American Server. Once you have a 'console' then issue the command -->> iptables --flush --<< This will remove the rules from the IP tables which block requests to the Server from being replied to. Instead of trying to learn about the 'iptables' program, I am going to do a complete fresh install of slide and attempt to get it working in standalone mode, with a simple file store. In order to do this, I am going to use ... (?) * 2 june 2003, monday Electricity to the house in Ametlla not working until 4pm approx. Fuse was the problem. The fuse was broken in a way that was not usual. That is the fuse was not 'blown' in the normal way, but was broken in some other way which caused sparks to come out of the fuse box * 3 june 2003, tuesday looking at iptables still. I think the problem was not really an iptables problem. The problem which I am referring to is the American Server blocking attempts to connect to it from the Laptop computers located in Ametlla de Mar. I believe that this was actually caused by a program called 'portsentry', although I am still not entirely sure. I have temporarily stopped this portsentry program until I have time to understand it more fully. I believe that the portsentry program dynamically adds entries to the 'filter' table of iptables. It does this in order to block any ip addresses which it considers to be 'attacking' the American Server. However, any request to a port such as 3306 (the MySql port) was also being regarded as 'aggresive' or hostile. * 4 june 2003, Wednesday I am doing some testing and documentation of the Alpha version of Alexis. To this end, there are several related tasks which may be considered useful. Write a script which backs up the MySql database in which all the documents are being stored. With luck I should be able to do this with an ftp script (if my Microsoft Windows ftp program allows me to 'shell out' during a scripted session, using something like -->> ftp>!tar -czvf /mysql/data/dir/ --<< Also, I could write a script on the American Server which 'pushes' the data onto the Barcelona Debian Server. Write scripts which convert my current documents into a very simple kind of DocBook. I could do this by modifying the scripts plaintext2html-forum.sh and linkdoc2html-forum.sh Work out how to change the schema and style-sheet for Alexis Learn a little about the normal usage of Docbook tags, so that I can use them effectively. Think about the command line client for slide, so that I can script additions to the slide repository (MySql Database). Will this client run on windows? * 5 june 2003, Thursday Made a script that turns a plain text file into DocBook XML. Basically all the script does is mark up 'section headings' which are all capital lines, mark up paragraphs, which are seperated by blank lines, mark-up urls which start with http:// or something similar and other various simple things like this. * 6 june 2003, Friday Having a look at the slide command line client. The start-up script for the client does not work on the American Server, but the client can be got to run by executing each of the lines within the script individually. Not sure why the script won''t run. It appears to be acting under a different command interpreter (ie not sh and not bash) * 7 june 2003, Saturday Some testing of Alexis and documentation. * 8 june 2003, Sunday nothing * 9 June 2003, Monday Searching for a better Docbook XML stylesheet. Found one with the XMLmind editor. Thinking about wikis again * 10 june 2003, Tuesday Stuff to do: get a Xalan transform for DocBook:yep, get cocoon working with the slide version of tomcat:sort-of, make the Docbook css stylesheet. using the various files with the XML mind editor:yep, get a better WebDav client: something that can be scripted to put a file up on the webdav server. Below is the link to the Docbook XSL style sheets, to be used with Saxon, or with xsltproc http://sourceforge.net/project/showfiles.php?group_id=21935 In the end I was not able to install a scriptable webdav client: see the document http://www.ella-associates.org/alexis-info/docs/webdav-stuff.txt for the results of my endevours to find something suitable I uploaded the new DocBook Style sheet, which appears, from a very cursory glance, to be working. * 11 june 2003, Wednesday Am looking at the cocoon configuration problem. To start and stop tomcat webapplications without starting and stopping the server at a user with role 'manager' to the 'tomcat-users.xml' file then stop and start Tomcat, then access the url http://ella-associates.org:8080/tomcat-manager/list And log on with the user name and password which you have just entered This will provide a basic interface to starting and stopping webapplications as well as installing new ones. * 12 june 2003, Thursday Will work on making a backup of the mysql (slide) database. Importing all existing documentation into Alexis (slide), and establishing 'cocoon' transformations. The XSL stylsheets appear to be 'carking it' when, for example, the alexis-faq.xml document becomes complex. That is, when sections were added to the document each containing a qandaset, the stylesheets failed to render the document into HTML. Only the table of contents was produced and nothing else. * 13 june 2003, Friday, Need to move the DNS configuration onto the American Server. * 14 june 2003, Saturday, Nothing * 15 june 2003, sunday Deal with Roberts email problem. Do the database backup. Deal with the DNS problem. Wrote a backup script for the mysql slide database, and scheduled it for every 2 hours, since the script runs extremely quickly and the database is not big. About 80 unique versions will be kept * 16 june 2003, monday Local Server has gone down. Working on DNS Migration. * 17 june 2003, tuesday Achieved DNS migration. The keys to using DNS are doing 'rndc reload' when you change the configuration files and do 'rndc flush' to get rid of the DNS 'cache'. There are only two important configuration files namely 'named.conf' and the configuration file for each 'zone' which you control. Actually, this isn't quite true but the other configuration files you can pretty much leave as they are. To test a DNS configuration use 'dig' with the '+norec' option which stops dig from going ahead and doing all the work by itself. so just do something like this -->> dig +norec hostname --<< There are important things to remember about DNS resolving. Remember 'who' is doing the resolving. That is, which name server is doing the lookup. If you dont tell 'dig' which nameserver to use or to query, dig will use the one in the file 'resolv.conf' To test a DNS configuration use 'dig' with the '+norec' option which stops dig from going ahead and doing all the work by itself. so just do something like this -->> dig +norec hostname --<< There are important things to remember about DNS resolving. Remember 'who' is doing the resolving. That is, which name server is doing the lookup. If you dont tell 'dig' which nameserver to use or to query, dig will use the one in the file 'resolv.conf'. The thing is, do nameservers resolve anthing or do they just refer you to other name servers if they dont know the answer. I dont know. I am still slightly confused. I think it is that the first name server which you make a request to actually does all the hard work of querying other name servers. * 18 june 2003, wednesday stuff * 19 june 2003, Thursday stuff * 20 june 2003, Friday Emailed Erik seth about programming. Erikseth is going to try to hellp us in the Alexis programming effort * 21 june 2003, Saturday Worked on the Posidonia web-site. This is available currently at the URL http://poseidonia.ella-associates.org/ * 22-26 june 2003 Not much * 27 june 2003, Thursday Poseidonia web site * 28-31 june 2003, Posidonia web site. Made the web-site into a wiki so that it can be contributed to by anyone. This may be useful in eliminating language errors in the web-pages. This is achieved by a script called text2html-collab.sh This script essentially creates a 'wiki' system but it is different from the normal wiki systems which are available on the net in that the 'editing' box is on the same page as the web-page and also the wiki 'language' is simpler than other wiki languages. This Posidonia Wiki has problems because the Browser does not refresh the web-page so that it looks to the web-visitor as if they havent done anything at all. * 2-6 june 2003 Made indexes for the Alexis documentation using cgi transforms of the slide web-diretories. These transforms point the links to the required place which is http://www.ella-associates.org/cocoon/livedocs/ etc The cgi scripts are currently located in the /var/www/alexis-info/ directory. I am not entirely sure why I put them there. I then reduced the 'indexing' scripts to one script which does the work for all the directories and I have included aliases in the apache configuration file so that if new directories are added to the documentation they will be automatically 'rendered' as they should I also rearranged the Alexis documentation into directories such as programming, alexis-overview, alexis-help, e-learning, business, general. The first 5 of these were suggested by nick but I added the last for the things which didn't fit into any of these. Enhanced and generalized the Posidonia web-site wiki. * mid june 2003 Helping Marc Vila i Leon with various posters and flyers for posidonia * 15 june 2003 Helping Erik-seth with his introduction to programming Yesterday it was reported by Juanma (Joan Manual) a resident of L'Ametlla de Mar that the Posidonia music festival website was not available when he tried to access it. I will investigate. Yes, all requests to the ella-associates.org web site seem to be failing now. Not good. Could this be related to the Cocoon aliases which I created. Yes I think so, because I didn't put the aliases in the virtual host. Because the American RedHat Apache web-server now has multiple virtual hosts, many directives should be put in between the tags. To do this you can use the alias which I have defined; namely type apache.conf and go to near the end of the file. directories. Yes this was the problem. I really need some kind of daemon script to alert me when one of the web-sites goes down. Strangely when I removed the aliases requests to http://poseidonia.ella-associates.org/ still did not work for a little while. The source tree for Alexis is at \\Lap\e\cvs\ella.org\alexis\src on nicks computer * 16 june 2003 Vim is still 'auto-wrapping' lines on some occasions. This is despite my attempting to fix the problem many times. The solution may just be to set the textwidth to 0. One of my tasks is to set up cvs on the American Redhat Server. This involves making sure that the text files have the correct line endings (since I ftp'd them from my Almetlla laptop as a .tar.gz binary file) informing cvs what are binary files (see april 10), importing the code tree, setting up cvs 'modules' and restricting access to the 'pserver' * 17 july 2003 Teaching Eriketh about java programming and XML. We covered baic concept about classes Today the American Redhat Server stopped functioning. The server was unavailable for about 24 hours. Eventually when the problem was solved the technical support man at the server hosting company informed us that the problem was due to 'iptables' This is a 'firewall' program which is designed to improve the security of the American server. Ryan Flowers the technical support man indicated in his email that he has now disabled this program. He suggests in his email that we can see what he has done by running the 'setup' command. Nick is keen to put in place a 'physical reboot' device which would allow us to start the Redhat box if something goes drastically wrong. When the Linux Box restarted we had difficulty starting the James Mail Server. Nicks method of doing this is to use the 'telinit' command to change the 'run-levels' on the computer. Apparently the point of this is to start the Mail Server as a true 'service' rather than as a process that is owned by a particular user. Specifically Nick executes -->> telinit 4 telinit 3 --<< Although it may be the other way around. The problem with this was that the sendmail daemon also has scripts in the /etc/rc.d directories and so when the commands above were executed the 'sendmail' daemon actually 'grabbed' the relevant email ports (eg: 25) before the James mail server. This meant that James was unable to start properly. This occured because sendmail is written in C whereas James is written in Java and therefore the sendmail daemon carries out its work slighly faster than the James Daemon * 18 july 2003 teaching Erikseth about Java programming. Nick was keen for us to write a 'document change notification' program, but in the end we did not have time The mentioned program would send an email to somebody when a document was changed through Alexis. The idea behind this is to notify editors of documents when a document they have been working on has been modified by somebody else. The program should be reasonable simple to write since in doesn't need to be integrated with the rest of the Alexis code. It would need the following components: jdbc to access the slide database and decide which documents have changed; javamail section to send an email to those people who have an interest in the specified documents. Nick would like all the database calls to be in one Java class in case the database schema changes. Originally Nick wanted this Document-change-notification element to be written as a JSP running in the Tomcat Servlet container. This would allow the program to have a web-interface or perhaps just some kind of web 'activation' component. * 19 july 2003 I am in Barcelona. My keyboard on the laptop is now not working very well. This is due to the fact that during the journey from L'Ametlla de Mar to Barcelona the various bits of 'stuff' (dirt) which is floating around under the keyboard have moved around and lodged in different places. In particular the s and x keys are particularly bad. This can normally be rectified by blowing forcefully in the direction of the problematic key with your mouth close to the keyboard. Did not do much work. Saw paul andersen in Barcelona. Keyboard problems fixed by blowing method * 20 july 2003 Need to complete cvs setup. Do we really need backups every two hours for the slide mysql database? Changed slide database backups from every 2 hours to every 6 hours The Local Server (based in calle Blanqueria) is now permanently switched off. Went to picnic with Robert * 21 july 2003 Need to look at why Cocoon is not dealing with European characters properly. This was noticed by Nick Reddel. Also, need to edit Nicks business proposal Had a meeting with robert about Alexis stuff. Wrote to American Server people about putting in a 'physical reboot' function which would allow us to actually turn the American Redhat Linux Server on and off and therefore debug problems with the Server if they arise The name of the main technical support guy on the amhosting server is Ryan Flowers and his email address is ryan.flowers@amhosting.com * 22 july 2003 The American Server went down again today. The American Server company claim that it is our software which is the problem. The James mail server has not started properly * 23 july 2003 To start james do the following Make sure that James is stopped by doing /usr/bin/james-2.1.1/bin/phoenix.sh stop Kill all sendmail, poprelayd and maybe 'master' processes by doing ps -el to find out the process numbers and then do kill number to get rid of each process Now do: telinit 4 and then telinit 3 in that order to start james as a service. Now check that james is running by doing james.manage or telnet localhost 4555 to connect to the 'manager' application Below is a line which could be used to automatically kill sendmail type of things automatically -->> for n in $(ps -el | awk -F" " '/sendmail/{print $4}'); do kill $n; done --<< * 24 july 2003 I created a script at /var/www/utils/start-james.sh which does all the above mentioned processes of stopping james and changing runlevels, etc. We have been experiencing ongoing problems with the American Redhat server. On one ocasion today Nick issued the command iptables --flush in order to attempt to allow access to the Apache Slide Server which carries out most of its transactions on port 8081, but this command actually killed the Server, or at least the Server died simultaneously with this action. However, the 'physical reboot' feature has now been implemented on the American Amhosting Redhat Server and by telneting to a particular port of a particular ip address it is possible to restart the Server. This was used to over come the problem -->> iptables -A INPUT --protocol tcp --sport 1024: --dport 8079:8084 -j ACCEPT --<< The line above is, according to Nick, the correct line to flush the 'iptables' on the Server and thus allow necessary access to the Server. And here are the details for accessing the reboot feature -->> Email from Ryan Flowers of Amhosting, Reno America I added this and billed your card for the $59.90 required for it The reboot switch is internet accessible and turns off/on the power switch To login, telnet to 63.105.72.10, username nick, password . This will add 9.95/mo to your account --<< According to Nick this works without a problem The afternoon and Evening: I worked with Robert Grundy looking at rival xml editors to Alexis and talking about what features make Alexis important. Robert made a number of, in my opinion, very valid comments about the Alexis interface. That all 'functions' or actions should be available from the menus. For example the list of valid tags for insertion into an XML doc should be available from a menu. That it is tricky to get the cursor in between an empty tag set. Really the 'triangles' of the xml tag markers should not be 'sensitised'. That is, if you click on the triangles then the cursor should be put in between the tags rather than selecting the set of tags. Also: it would be nice if the mouse pointer would change its shape or symbol in the sensitised areas. That is, if, by clicking, you are going to select the set of tags then the mouse pointer should be a certain symbol but if by clicking you are going to put the cursor between the tags then the mouse pointer should be a different symbol. This is in accordance with the Xmetal editor That the XMetal editor starts up in less than a second whereas Alexis takes about half a minute. That when you gain a lock on a file by double clicking on the file name and then editing that document, then really you have 'checked out' that document, and for that reason you should have the option of 'checking back in' the document manually, rather than just be saving it. This point may be dubious. However the menu item for 'check in' is disabled when the document has been automatically checked out. That if you are not connected when you try to login in using Alexis then the operation system connect function (window) should be triggered. This may be tricky to implement. That you should be able to edit the schema document, if you have sufficient authority. End Of ROBERTS SUGGESTIONS Nick has suggested: move cgi scripts to Java, write the email notification thingy, work with Robert. Also I should stop James being used as an open Relay for spam. This is probably super super important but I am going to bed. Email sent from james account to the outside world DO NOT arrive. not good The Server appears to have stablilised. * 25 july 2003 mail server still down to stop it being used as an open relay for spam. I could write a script which looks at the james mail spool (outgoing queued mail) and if it becomes very large this script could send an email to alert somebody that the mail server is being spammed. I wrote this script and called it /var/www/utils/check-spam.sh . It can be run from anywhere with check-spam.sh but the point of it is for it to be scheduled using crontab to automatically send an email if the server is being spammed. I have scheduled it in this way. Currently it is every five minutes but I will reduce this soon Also a script which checks the webservers This James Mail Server spamming problem may be fixed but I am not entirely sure about this. I changed the configuration by adding the extra authentication check that the user should have the correct email address. Also I wrote a lot of aliases for the james server to make it easier to see what is going on in the log files. Type alias | grep james to see these aliases From looking at the logs, somebody seems to have the 'webmaster' password. Although they might have others. Found a thing called jvmail which is a java console smtp client. Could solve all my email problems. I couldn't get the latest version of JVMail to work, but an older version does.:q Here is the command line which seems to do the trick java JVMail -nmx1.mail.yahoo.com -ffrom@test.com -tmatth3wbishop@yahoo.com -s"test jvmail 1.1" -b"Message Body" * 26 july 2003 Early this morning I was investigation simple command line smtp clients. and I found certain things but I did not have great success. For example jvmail is a java smtp email client which appears from the website to have all the required features. But I couldnt get the latest version to work, since it uses a jar file and all my attempts to run the jar file using things such as java -jar JVMail.jar all failed. Also it appears that the 'manifest file' in the jar did not contain the necessary configuration lines There is also a c program called something like SMTPclient on the internet. But it doesn't contain any compiling instructions. I should be able to overcome this but it was a bit intimidating Also there would be various perl scripts progs to do this but I cant seem to get my perl ::net packages working. All in all the conclusion was to use the JavaMail api to do this myself. This would make sense because I also have to write the 'alexis notification' program which would have to use the javamail api. The James Mail Server does not appear to be being spammed anymore. The spammer is still attempting to connect from 218.70.141.188 but cannot because I changed the webmaster user account password using the James 'remote manager' * 27 july 2003 The James Mail Server has gone down again. I am investigating the cause. Server restarted. I dont know why it stopped. Maybe sendmail again * 28 -31 july 2003 pottering around with various things, not much really. saw michael the cellist about a room in his flat. * 1 august 2003 Probably should start on the email notification script. The ftpserver on the American Server does not appear to be running Beginning to look at the 'javamail' api. I think that I am going to have to install this. See the file http://www.ella-associates.org/alexis-info/docs/javamail-stuff.html * 2 august 2003 Understanding JavaMail and the Slide Webdav java apis I wrote a program at http://www.ella-associates.org/utils/TestJavaMail.java which successfully uses the javamail application programming interface (api) to send an email using normal smtp authentication work time: 2 hours [3] * 3 august 2003 Wrote a set of java programs GetDavFile.java PutDavFile.java and also vimdav.sh The vimdav.sh script allows you to edit a file on the webdav server using the 'vim' text editor. This script uses the two previous java programs. These can all be found at http://www.ella-associates.org/utils/ -->> vimdav.sh /slide/files/documents/alexis_docs/the-sand-box.xml --<< The script has been invoked successfully with the invokation above. I am really quite proud of this since it makes making small changes to an xml document much simpler work time 3 hours [6] * 4 august 2003 Installed catdoc which seems to work well for converting Microsoft word docs to plain text, and also installed word2x which is supposed to do the same thing. But not for me Started to transform Roberts Glossary using the following command line -->> catdoc robert-glossary.doc | expand | \ sed -e "s/^[ ]*//g" -e "s/[ ]*$//g" | \ sed "/[ ]\{2,\}[^ ]/ s#\(.*\)[ ]\{2,\}\(.*\)#\1\2#g" | less --<< Had a 'business' meeting with Nick Reddel, Marcus and some girl. This meeting in reality was actually a series of beers and carajillos. The line below is nicks solution to the portsentry application adding lines to the iptables on the RedHat American Server which block us from using ports such as the webdav port I am somewhat dubious about this solution. Firstly it doesn't address the problem of portsentry Secondly if there is a reject rule in the iptables database already, is an accept rule going to do any good -->> iptables -A INPUT --protocol tcp --sport 1024: --dport 8079:8084 -j ACCEPT --<< However, this line above did just work for me tonight. I executed it and now I am able to get webdav access again. I wonder if my javawebdav scripts will work now. YES YES, my webdav java programs now work on Microsoft Windows as well as Redhat Linux. To make this happen I had to copy the necessary jar files to my MS Windows computer and then reference them with the classpath variable. I did not need to change the jar files in any way Here is what the windows invocation looked like. -->> D:\davscript>java -classpath d:\davscript\commons-httpclient.jar;.;d:\davscript\webdav.jar GetDavFil e files --<< work time 3 hours [9] * 5 august 2003 Had another business meeting with N Reddel and Marcus in the Galician bar which is near the Arc de Triomf. In this meeting Nick suggested the following work for me. finish the email notification of document changes withing Alexis. Mock up a structure for the Alexis (product) website using docbook XML. This would require an enumeration of the various categories and pages which would be necessary on the website. some suggested pages were: a comparison of Alexis with various other XML editors. A technical resources page, what ever that means, support, javadoc, general documentation. Look at Nicks mock up of the Alexis gui interface and make comments. Work on the server extensions to slide and webdav. This would be a major programming task but would, in my opinion be much less difficult than learning about the Netbeans architecture Tasks which I thought of myself were to do some general research on lexicographical companies in Spain. Also had the idea of doing some general 'training' in these companies which would essentially involve suggesting alternatives to Microsoft word As discussed, thee distinguishing feature of Alexis in document management, including change tracking and versioning etc. Also a messaging system and a 'plug-in' schema semantic help [10] * 6 august 2003 Nick has suggested that we need to solve a problem with the Cocoon Server in that it is not performing transforms on documents which have European accented characters in them. It is possible according to my small knowledge of this topic that the accented characters need to be 'entitized'. I have been able to insert some accented characters into some documents and been able to view transformations of those documents. I had a dicussion with Nick Reddel about my continued employment with Ella Associates. The following things were discussed: my disagreement about some of the processes through which Alexis is being developed. In particular the idea that my putting intensive effort into Alexis in the next 3-4 months we can in some way hit a 'jackpot'. My preference is for a sustainable development process, building the application up in useful stages, each of which 'stage' would represent a useable product which could be shown to companies. It is possibly to late to suggest this since Alexis has not been begun in this way an therefor its 'architecture' probably does not really lend itself to this. Nicks argument is that he does not have sufficient time or money to develope the application in this way. This argument is based on the idea that Alexis has a limited 'shelf life', or that if the application is not got out in a reasonable amount of time (within 3 months seems to be a very approximate figure), then there will be no market for the application. I disagree with this. I believe Alexis has a much longer useful working life than this. Nick mentioned the fact that he needs to be given estimated times for when certain tasks will be completed by me. * 7 August 2003 The file containing the problem accented characters was locked (in webdav) by nick. I am developing a webdav debugging java program ( http://www.ella-associates.org/utils/DebugDavFile.java ) which is designed to give information about WebDav resources located on the Apache Slide Server work time 1 hour [11] * 8 august 2003 reformatting shell scripts to java in vim -->> % s/elif/} else if/gc % s/;[ ]*then[ ]*$/ {/gc Format boolean variable comparisons % s/if \[[ ]*"\$\(b[^"]\+\)"[ ]*\(!\?\)=[ ]*"true"[ ]*\];\?/if (\2\1)/gc Turn null script parameter checks into argument length tests :% s/if \[[ ]*"\$\([0-9]\+\)"[ ]*=[ ]*""[ ]*\];\?/if (args.length == \[\*\*\1 - 1\*\*\])/gc Turn non null parameter tests into java comparisons :% s/if \[[ ]*"\$\([0-9]\+\)"[ ]*=[ ]*"\([^$"]*\)"[ ]*\];\?/if (args[\1].equalsIgnoreCase("\2"))/gc Format string variable comparisons handling negative tests as well :% s/if \[[ ]*"\$\([^"]\+\)"[ ]*\(!\)\?=[ ]*"\([^$"]*\)"[ ]*\];\?/if (\2\1.equalsIgnoreCase("\3"))/gc And handle comparisons of two string variables :% s/if \[[ ]*"\$\([^"]\+\)"[ ]*\(!\)\?=[ ]*"$\([^"]\+\)"[ ]*\];\?/if (\2\1.equalsIgnoreCase(\3))/gc End of if statements to c/java curly braces :% s/\/} //-- if/gc Add a curly brace to isolated else statements :% s/\[ ]*$/} else {/gc Make thens into curly braces :% s/\/{/gc Make embedded variables into string concatenations :g/echo[ ]\+"/ s/\$\([^0-9][a-zA-Z-]*\)/" + \1 + "/gc Make echo statements into string buffer append statements :% s/\ and <92> in the createca document do not seem to be the problem. The cocoon logs show that a UTFDataFormatException is being thrown by this document. fixed the createca document accent problem Alexis actually uses the iso-8859-1 character set. but has the meta directive At the moment, accented characters entered into Alexis will look alright in Alexis but will cause cocoon to crash because it is expecting utf-8 accented characters, not iso-8859-1 accented characters The simplest solution to this is to change the meta directive of the document to This needs to be done for every document which may contain (iso) accented characters The only way I know how to do this in Alexis is through the 'source editor'. Is there another way? I suppose this should be the default for all documents until alexis displays the utf character set properly. Another solution is to make Alexis display utf properly and to convert our existing documents to utf This can be done on the server with the command iconv -f iso-8859-1 -t utf-8 -f newfilename oldfilename For the createca document I changed the meta directive and it now displays correctly in cocoon and alexis Added pdf and plain text transforms to the Cocoon sitemap.xmap file. This means that live renderings of Alexis files can be seen in plain text and in pdf formats. The relevant sitemap is the one in the cocoon home directory. There is an alias cocoon.alexis.sitemap to access this file Discovered a thing called apache forrest http://xml.apache.org/forrest/ which is built apon apache cocoon and which seems to include a nice looking wiki called JSPWiki. http://www.jspwiki.org/Wiki.jsp Forrest may not be a mature product work time 7 hours [35] * 11 august 2003 1 am: 2 hours in early morning 2:30 pm investigating a website system for the Alexis project. Am thinking about using the Apache Forrest system. I set up the forrest system on the ella associates server. It can be seen at http://www.ella-associates.org/alexis/ Worked to 6pm time 5 hours worked from 10:30 to 3:30 Nick way to open a hole in the iptables for the Mysql database -->> iptables -A INPUT --protocol tcp --sport 1024: --dport 3306 -j ACCEPT so i can look at mysql from 'ere --<< Nick has requested that the cvs be set up I got external document feeds working in Forrest. Firstly, copy the sitemap.xmap file from the build/tmp directory of forrest look at the document http://xml.apache.org/forrest/your-project.html#integrating_rss now copy that code in the 'internal URI space' section of the sitemap.xmap file. Change the transform to 'docbook2document.xsl' etc. Place 'body-' before the apparent file name, this is to allow the skinning process to work. The only problem now is that the docbook2document.xsl style sheet doesn't seem to work very well. But this can be solved using the normal docbook style sheets. See the forrest faq document for how to do this . total hours=12 [47] * 12 august 2003 worked 11:30 to 3:30 trying to fix things in the Alexis website at http://www.ella-associates.org/alexis/ but with only partial success. I think that I need to download a more recent version of forrest using CVS. I will also set up the Ella CVS system today. Have to meet michael the cellist to help him with dome midi file problem at 6pm total hours: 4 [51] * 13 august 2003 I saw michael today and discussed moving in an making copies of CDs. started work 4:30 writing emails Trying to get the cvs wrappers list of the local computer, but using PQBoot managed to make the computer unbootable. It was booting automatically into windows, and now is always trying to boot from ide0 no matter what the cmos settings are. I made a list of binary filename extensions for the cvswrappers file by using the following line -->> lynx -dump http://www.ace.net.nz/tech/TechFileFormat.html | \ expand | sed "s/^[ ]*//g" | \ sed -n "/^[A-Z0-9]\+ /{G;s/\([A-Z0-9]\+\) \([^\n]*\)\(\n\)/#-- \2\3*.\1 -k 'b'/gp;}" | \ tr '[A-Z]' '[a-z]' > cvs-binary-list.txt --<< And I made a set of uniq file extensions and what they mean in the alexis source tree with -->> for f in $(find . -iname "*.*" | sed -n "/\.[A-Za-z0-9]*$/p"); do file $f; done | sed "s/^.*\(\.[a-zA-Z]\+:.*\)$/\1/gi" | sort | uniq | less --<< worked till 8:30 pm total hours 4 [55] * 14 august 2003 There is a fiesta in the local tonight which will probably stop me doing any work. El cumpleanyos de Victoria * 15 august 2003 nothing * 16 auguste 2003 some cocoon forrest work. Cocoon is giving 'out of memory' errors. Changing the main sitemap back to the old version did not work 1 hour [56] * 17 august 2003 4:00pm a bit of cvs. thoughts: in my opinion there really should be email notifcation scripts for the following; cocoon, slide, james, Apache (including websites), dns mysql, forrest, and the actual American Box itself. Also notification scripts for disk usage on the American Redhat server Further struggles with cvs. I created a hole in the filewall with this -->> iptables -A INPUT --protocol tcp --dport 2401 -j ACCEPT --<< And I made an xinetd file called /etc/xinetd.d/cvspserver based on info in the cvs faq then I restarted the xinetd server with /etc/init.d/xinetd restart Then I tried to make an anonymous user in the 'passwd' file, now I just have to make some real users using a perl trick also in the FAQ, and things probably should work. Also see my comments about setting up CVS the previous time- some months ago. (Search for cvs in this document) Then I needed to update the permissions on the cvs repository with something like -->> cd $CVSROOT/alexis; chmod -R a+w * --<< The R switch makes this a recursive change. However we now have a different problem, namely that it doing a checkout from the cvs repository a 'disk quota' was exceeded. I am not sure where this disk quota is coming from. Is it part of the 'ftp' system (which has nothing to do with CVS) or part of the Unix system, or is it a problem with my local computer. worked till 7pm worked 2am-3:30 total 4:30 hours [60] * 18 august 2003 1:00pm some work. I still need to finish off the cvs setup * 19 - 21 august 2003 I havent been doing work for ella associates during these days. I fixed some 'midi' files for Michael and met various people. -->> for f in *.sh; do g=$(echo $f | sed "s/\.sh/\.txt/g"); upload-geocities.sh $g eg/unix-shell ; done | less --<< a snippet to upload stuff to a geocities account TIPS AND TRICKS AND INFORMATION use the 'top' utility to look at whats happening on the server. This utility shows which applications/ daemons are using up the most processing time. To start and stop tomcat webapplications without starting and stopping the server at a user with role 'manager' to the 'tomcat-users.xml' file then stop and start Tomcat, then access the url http://ella-associates.org:8080/tomcat-manager/list And log on with the user name and password (10 august 2003: ) which you have just entered This will provide a basic interface to starting and stopping webapplications as well as installing new ones. http://www.keysolutions.com/ServletFAQ.nsf/0/99abc10529dab92a8525686700737ea2?OpenDocument An example Java Servlet If you want to reload a .bashrc file or something like that type source .bashrc or something similar wget http://www.ella-associates.org/utils/ -nd -np -r -l1 -Rgz This is a command line to get all the files in a remote 'web directory' based on the index listing given by the web server. This example rejects files with the gz file name extension You can set the system clock with something like date --set='+4 hours' which will advance the clock four hours. Type 'info date' for more information The command below gives a 'nice' full text date format with weekday name and AM/PM time (no seconds) date "+%A, %d %B %Y, %I:%M %p" www.insecure.org The home site for the nmap port scanner and a good general security site. important directories /var/www/ the root for documents served by the Apache Web Server. A Command Line to send an email through James using 'Blat' (on Windows). This command sends whatever message the user types at the command prompt (ending with ^Z) and displays the responses of the server. blat - -t matth3wbishop@yahoo.com -plain -u matthew -pw bacibaci -debug A simple example of an nroff (groff) document is available in the 'vi.advanced' vi tutorial document. Also included is the command line to process the nroff document. to connect to the Alexis Server on the 'local computer' use the url string www.ella-associates.org/alexis/alexisServer to restart the inetd server, (according to a cvs mailing-list archive) you can type 'kill -HUP ' where the stuff in angle bracket is not taken literally. Get the process id by typing 'ps -el' The 'ispell' program can do a spell check of html documents (and xml) using the -h switch. to get basic linux help type 'info Cookbook' to view mysql docs at the console type 'mysqldoc' XWindows KDE Font size configured in main menu --> control center --> look and feel --> fonts For setting up the DNS Bind package, see the Linux Network Administrators Guide. For a discussion about 'vi' and 'vim' the Unix text editors please see the document http://ella-associates.org/alexis-info/docs/important-tools.html Also included is a list of vi commands. A url to send text messages begira.com The account is u=nitsk p=ouiji But I have only managed to get the message 'mensaje no enviado. Codigo 3.' I dont know what 'codigo 3' stands for. There is some suggestion that it is possible to send SMS messages through a thing called ICQ. http://www.mobilesms.com/whatis.asp A site explaining SMS from a technical and other point of view Some classical music sites A schubert impromptu, chopin etc http://www.iuma.com/IUMA/Bands/beatasza/ Nice quality, interpretation? http://www.classicalarchives.com/artists/indexa.html#s big collection, mainly midis, dubious quality to restart the local server: resue root=/dev/hdb4 and then run fsk if there is some hard disk problem