Can you import a non-forum database into SMF?

[Duffield1]

I run a couple of SMF sites, which function wonderfully at sites per se (and thanks to all those that contributed to making the software so effective!)

My next project is to convert a legislation information source into a searchable forum-type site - I have about 2000 text files which I want to import into SMF effectively as posts attributed to a particular user.  The source information follows a tree-type structure, which SMF lends itself to perfectly, and the subscription levels (paid, regular renewal) and permissions make it ideal for my purposes.  However, is there any way, other than copying and pasting all the text files as individual posts, to do this? 

A friend suggested that they could all be imported into a Microsoft Access database (so I could put items into specific fields, for example), but would it then be possible to import these into an SMF database?  Any hints and tips (or even a "don't even think about it!" message) are welcome!

[Kays]

Since they are .txt files, you should be able to upload them to your server and then write a script to loop through them, get the contents and use the createPost() function to add each to the SMF database. You will need to include SSI.php for this to work.

Do note though that, 2,00 is a large number of files to be working with in one folder.

[Duffield1]

Thank you!  That sounds wonderfully simple, and I can cope with the uploading to server, but I start to struggle a bit with the writing of the script!

I'd want to move them around into different boards on the forum, but am happy to do that manually.

How on earth would I even start writing an import script, or could I pay someone to do that?

[margarett]

We would need to see one of the files structure. Shouldn't be too complicated, I imagine (or just dreaming high )

For you to place them automagically (more or less) in certain boards, the best way would be for you to place them in folders named with the ID of such boards.

[Duffield1]

I love to dream high!

Here (in italics) is the content of one of the files.  I've attached the original file, too.  They are all rich text files, so the title (which would be the 'subject' line in the post) is always underlined.  I should add that there is not supposed to be a smiley in there!

SI 2001 No.759
The VAT (Electronic Communications) (Incentives) Regulations 2001

[Note: This SI was revoked with effect from 22 July 2004 by SI 2004 No.1675]

Citation and commencement
1. These Regulations may be cited as the Value Added Tax (Electronic Communications) (Incentives) Regulations 2001 and come into force on 1st Apri12001.
Interpretation
2. In these Regulations-
"the Commissioners" means the Commissioners of Customs and Excise;
"electronic return system" has the meaning given in regulation 25( 4B) of the VAT Regulations;
"incentive payment" has the meaning given in regulation 3 below;
"prescribed accounting period" has the meaning given in regulation 2(1) of the VAT Regulations;
"relevant return" means the return referred to in regulation 3(1)(a) below;
"taxable person" has the meaning given in section 3(1) of the Value Added Tax Act 1994;
"VAT Regulations" means the Value Added Tax Regulations 1995.


Incentive for making a VAT return by way of an electronic return system
3.-(1) The Commissioners shall give an incentive in the form of a payment of ;£50 to any taxable person who-
(a)   makes a return by way of an electronic return system on or after 1st Apri12001;
(b)    has not previously made such a return on or after that date; and
(c) satisfies the conditions specified in any direction given by the Commissioners under paragraph (2) below.
Such a payment shall be called an "incentive payment".
   (2) The Commissioners may give a direction specifying any or all of the following conditions-
(a) that the relevant return is made in compliance with any time limits specified by or under the VAT Regulations;
(b) that the relevant return is made for a prescribed accounting period that is specified in the direction;
(c) that the amount of value added tax that is payable in respect of the period to which
the relevant return relates is paid-
(i) by a means of electronic payment that is specified in the direction; and
(ii) in compliance with any time limits specified by or under the VAT Regulations;
(d) that the value of supplies made by the taxable person in the period of 12 months ending on the last day of the prescribed accounting period to which the relevant return relates does not exceed such sum as may be specified.
Withdrawal of an incentive payment and recovery r
4.-(1) Where it appears to the Commissioners that a person has failed to satisfy one or more of the conditions specified in a direction given under regulation 3(2) above, they may give notice in writing to that person withdrawing the incentive payment and, where a payment has already been made, may assess the amount so paid and notify it to him.
   (2) The Commissioners may notify an assessment under paragraph (I) above at the same time as they give notice withdrawing the incentive payment.
   (3) Subsections (2) to (4) and ( of section 78A of the Value Added Tax Act 1994(a) (assessment for interest overpayments) apply in the case of an assessment under paragraph (I) above as they apply in the case of an assessment under section 78A( I) of that Act.
Appeals
5. An appeal shall lie to a VAT and duties tribunal with respect to-
(a)   an assessment under regulation 4(1) above, or the amount of such an assessment; and
(b)   a decision that the conditions of entitlement to an incentive payment are not met.

Directions

6. Any direction given by the Commissioners under regulation 3(2) above shall be contained in a notice published by them.

[margarett]

Hummm that would be more difficult, because it would be tricky to read RTF files (but possible, so uncle Google says http://webcheatsheet.com/php/reading_the_clean_text_from_rtf.php )

[Duffield1]

Or even with a bulk conversion of rft to txt?
http://www.emreakkas.com/localization-tools/convert-rtf-to-txt

What do I need to do to commission a script?

[margarett]

Later today I will try to build an example.
If someone knows how, please go ahead and bring it Or else I will have another try-and-fail session again

[Duffield1]

Thank you! :-)

[margarett]

This is being kind of fun
Although I'm waaaaay over my capabilities, it's kind of fun to try and fail and to understand why I'm failing

Anyway, I got to the point where I read the file contents. And the function I indicated above is not quite good

Normal; heading 1; heading 2; Body Text 2; Body Text; Body Text Indent 2; Body Text Indent 3; SI 2001 No Mike Commins Mike Commins - Aries Accounting Services Ltd SI 2001 No.759 The VAT (Electronic Communications) (Incentives) Regulations 2001 [Note: This SI was revoked with effect from 22 July 2004 by SI 2004 No.1675] Citation and commencement 1. These Regulations may be cited as the Value Added Tax (Electronic Communications) (Incentives) Regulations 2001 and come into force on .....

So, no line breaks and, worse than that, some "not text" is read also... And, of course, I'm not familiar with RTF file format so it's quite hard for me to investigate what does this function do...

I also tried your "batch" conversion to TXT. That link brings me to a program that does not run in my sandbox, so I got another one:
http://download.cnet.com/Rtf2txt/3000-2079_4-10601914.html
And this works flawlessly.

But the thing is: ALL your files have a 2-line subject? Or, in the lack of it, a blank line between the subject and the "body"?
Because here's the deal: since there are no marks in TXT (therefore, the RTF would be better, if I knew how to handle it ), the division would have to be done in specific markers. With RTF we might be able to use the underline (heck, maybe even the text color!) to divide the content. But in plain text, it would have to be a specific divider...

So, as far as I see it: it's not THAT hard to pick some text and to put it as a new post by a specific user. The real difficulty would be to correctly interpretate the original format...
That's why this would be a specific solution for THIS specific problem.

For me, it's funny and I will do my best to continue to help you anyway I can. But, unless I get a "RTF-basis-in-a-nutshell" course , I need you to check that your source files comply with a specific and fixed structure so that can be handled as "simple text"

[Duffield1]

Just an idea, what if the files were converted to HTML instead of TXT?  Perhaps the formatting marks (<I> </I>) would be more obvious?

[margarett]

Did you find a batch converter for this? I haven't search for it, did search for XML but found nothing...

That would be a good idea

edit: just found this:
http://www.nchsoftware.com/documentconvert/index.html
And it seems to work

[margarett]

Are you still interested in doing this? I am on vacations these days but haven't forgot this

[Duffield1]

I'm afraid that I have abandoned all hope and am copying them manually - it is a big job, but it'll get done... eventually!