-  [WT]  [Home] [Manage]
I think Keine has the best hat. It's still darn silly though.

[Return] [Entire Thread] [Last 50 posts]
Posting mode: Reply
Name
Email
Subject   (reply to 7144)
Message
File
Password  (for post and file deletion)
  • Supported file types are: GIF, GIF, JPG, JPG, PNG, PNG
  • Maximum file size allowed is 2048 KB.
  • Images greater than 200x200 pixels will be thumbnailed.
  • Currently 890 unique user posts. View catalog

File 130676474350.jpg - (20.19KB , 300x300 , dc1ed1a614b4051ab0965e0ed0bad7a2.jpg ) [iqdb]
7144 No. 7144
Downloads the author’s posts (or all posts) of a CYOA thread or full length story in three file types; doc, txt, or the original html with images.

The link to it is http://westisblue.com.

It is pretty self explanatory; though I can post a guide if anyone really needs it. This thread is open to bug reports, feature requests and, if you feel like it, intelligent feedback. Demands and other stupidities will be ignored.

The software is in beta, so it may not work that well.

If the website goes down for content changes, I’ll make a post here or a redirect on the site saying so.

Quick Background:
Originally I just wanted to make a single thread downloader for myself, but feature creep set in and after a while, I felt like some people of the community would like to use what I’ve developed (if it works well enough)

I did some stress testing across a couple of computers, and the downloader pulled through (15 threads per two seconds on average), but the webserver I’m using is more finicky than my last one, so we’ll see if it can survive.

Please go see http://westisblue.com/?=faq for more specific information about the downloader which I don’t feel like retyping here.

Finally, though fixing bugs should take precedence, I’d like to know what else people would prefer in the downloader.

So without further ado,
[ ] Write in what feature you’d like to see me add.
Expand all images
>> No. 7145
Issue: Sometimes authors forget to post with their trip

Effect: Story Posts not included

Repeatable: Yes



Issue: Sometimes authors start with [x]Most Chosen Choice

Effect: Story Posts not included

Repeatable: Yes

Other than those two bugs for the moment, which I can't see an easy way around, it looks pretty darn nifty!
>> No. 7151
>>7145
as long as they include a recognizable postername, e.g. if writefag88888 forgets his 8888, it should still recognize it.

I think I can guess which process number you were, but please tell me it anyhow. Supplying process numbers are much simpler and easier than having people describe their inputs.

In the meantime, if the downloader is being excessively stupid in recognizing names, please just check all posts, and set smallest word count to like 500 or something.

EDIT: in response to the second bug, (if it exists, I'm not sure-- your output seems fine to me) precluding choices deletes all choice sentences but not the whole story post. if there are no choices for a story post (and they preclude author's comments), then there's a problem, but still, a fixable one.
>> No. 7152
[x] Would it be possible to have multiple address bars such that it's possible to paste a series of URL's in and thus download an entire story at once?

[x] Is it possible to include an output set to an ebook format?

Bug: apostrophes in download are replaced with a series of weird characters. Unsure if this is a browser glitch or if this is a problem with the converter itself.

This looks awesome by the way. Thanks for this.
>> No. 7157
>>7152
>[x] Would it be possible to have multiple address bars such..

Good sir just type in the story name (This shrine for example) and you will get what you're asking for.

>[x] Is it possible to include an output set to an ebook format?

I guess I could try and figure out how to do that (only if people vote on it though) But until then, the word documents is pretty versatile; you can probably batch convert the document output into ePub using other software.

with what sort of input are you seeing that apostrophe thing? I can't seem to see it.

also; you're welcome, I had some fun making this.
>> No. 7162
>>7152

ah I just put two and two together; its a browser glitch. Open the text file up in notepad and it should render correctly.
>> No. 7397
A little extra feature I put in. If a file was deleted from touhou-project.com, then the downloader will query its own database and convert that file instead.

Certain deleted stories still won't work (RAN) because the files for them in the database are not compatible. I may try to add functionality so it can process these files (or alternatively just convert them so they can work).

Due to other work/projects, I won't be adding much functionality that eats up alot of time, although you may expect small things like the story list to be updated from time to time.

EDIT:
When inputting an individual url, the downloader now queries its local db if it can't find the thread on touhou-project.

Also, by the time of this post, the downloader detects there are eight threads in the archives (17 including RaN- which the admins purposefully deleted) that have been arbitrarily deleted. The local db however should have most of these files intact.
>> No. 7410
For some reason, trying to convert a story with a ' in the title confuse the downloader.
I tried with Doll's Quest, and it failed.
>> No. 7411
>>7410
Correction. Doll's Quest isn't in Cthuloo's thingy yet. A Fairy's Tale works just fine.
Friction About Ground, who's recent too, doesn't appear in his downloader.
>> No. 7416
>>7410

You can input quotation marks and the like, and the downloader will automatically delete them and search for the story name without it. Simply put, having quotes won't affect anything.

>>7411

Correct on all counts. I'd prefer to use a dynamic listing of stories, but that's something (semantic/tagging) that should be done on touhou-project's side.
>> No. 7465
Local database of story files, names and authors has been updated to reflect the new story list.
>> No. 7556
This is totally awesome. You are awesome.

Suggestions:

[x] Option to concatenate stories on multiple threads into a single .doc or .txt.
[x] Option to output into .odt format.

First is because I'd like to create a single file that I can easily ctrl+F through to look find if I've already described something or used a bit of dialogue. I could do it manually, but if I changed my mind and wanted to look at a file with or without the voter's comments, I'd have to go through and piece a dozen threads together again.

The second suggestion is because though OpenOffice supports word documents fine, I'd rather they be in a native format if I'm going to manipulate them a lot.
>> No. 7560
>>7556
Your thanks is much appreciated.

As for your feature requests:

1. Sure why not? I can see why people would like this.

2. ODT format is doable, though that it would take a bit of time for me to learn how to manipulate the library.

If I get back to adding functionality to this thing, I'll see what I can do about your requests, and that ebook request from a while back. New directions to work toward are always fun.

One more thing, I fixed the ampersand when selecting a story for the person who has having trouble with that.
>> No. 7562
File 131035871876.png - (9.08KB , 344x135 , classy.png ) [iqdb]
7562
Well, I had a bit of free time today in between my writing, so I went ahead and added a bit of functionality.

Downloader now allows you to download whole stories as a single word, text, or html file.

There are little formatting things that could be cleaned up... but after seeing how massive these files can be, I don't envision too many people using this feature for normal reading.
>> No. 7569
That shit is awesome and so are you cthulhoo.
Thanks to you, maybe I'll be able to finish ASSM for good.
>> No. 7572
>>7569
beware the ending.
>> No. 7573
Stupid question: If I select to include all content in a thread, and html format, is it pretty much exactly the same end result as simply viewing the original thread in the story list archives?
>> No. 7578
>>7572
Yeah, I know Anon raged at the ending.
But I only heard rumors, so I'm expecting an "Usual Suspects" kind of plot twist.
>> No. 7582
>>7573
You'd have to set post count to 0 for that to happen. Otherwise it would just get the 'lengthy' posts.

>>7578
It wasn't a plot twist, I don't think, but hey, read it and see how you like it.
>> No. 7587
>>7582
I'd agree that it wasn't a plot twist, but it's undeniably a dick move and a poor example of storytelling. Especially if you consider the reason it happened at all.
>> No. 7829
Is there a way to save pictures with the text? Sometimes, they are relevant (Owen's Priceless Story, for example).
>> No. 7837
>>7829
Conceivably yes, although if I coded that in, I'd probably stick to thumbnails only (at least initially)

As it stands, images can only be viewed through html formatting when connected through the internet. If you guys want to see his suggestion implemented, please speak up/vote for it.
>> No. 7858
For some reason, with the Hakugyoukourou LA story, in the 5th thread, thumbnails aren't displaying properly. And clicking on them leads to a 404 error. I don't know if it's the thread's fault or the downloader one.
>> No. 7859
>>7837


Speaking up...
>> No. 7862
>>7858
It's working for me (just tried it out). I believe you were reading the thread when the thread/images on touhou-project were down. Try reconverting the story and if the problem persists, file a bug report according to the steps listed in the faq.

>>7859
Okay that's two votes, and a little bit more motivation for me to work on it, on top of my other work.
>> No. 7863
>>7862
Make that three.
>> No. 7866
File 131430638634.jpg - (163.29KB , 900x900 , f263523682b6f292af7352dc70a72d2d.jpg ) [iqdb]
7866
Make that fou-- oh wait.

V1.2 now lets you download thumbnails locally along with the story.

For html, it changes all the the thumbnail picture links to their local copies(so it doesn't need internet access anymore). For doc, it links/includes the thumbnails and for txt, it merely references the image name, e.g.: [103242455.png]

Initially, downloading with thumbnails will be slower (alot slower) than without. However this inefficiency is mitigated by how the downloader will cache the thumbnails for future use by other users.

Besides a bug fix here and there, I also updated the site design to make it look less awful.

Let me know if you notice any bugs.
>> No. 7868
File 131431434358.jpg - (157.94KB , 600x676 , Cthulhoo.jpg ) [iqdb]
7868
>>7866
If I could love a monster with tentacle making me crazy just by staring at me, I would.
Keep on with the good job, Great Old One.
>> No. 7873
>>7868
I feel vaguely heretical for being pleased by your thanks. Welcome all the same, as this update did take some time.
>> No. 7901
Local database of story files, names and authors has been updated to reflect the new story list as of Sept 1.
>> No. 7985
I tried downloading Expectation of Sanguine Disorder as a single file (1.32MB!). When opening in a web browser, it only shows until the end of the first thread. However, if I were to open it in notepad, it's quite obvious that the WHOLE story was downloaded. Any ideas why I can't read the whole thing in a browser?
>> No. 7986
>>7985
That's weird, I tried with another story, worked fine. Downloading as one file.
Did you checked hmtl formatting?
>> No. 7987
>>7986
Expectation of a sanguine disorder worked for me as one file. Try downloading it again. If there is still an error, please supply your process number in addition to a description of your error.

Side note: I think it may the most 'downloaded' story. Perhaps I should release some statistics later.

Another side note: the downloader now has a copy of the story list which is viewable via the main menu bar.
>> No. 7999
>>7987
>Perhaps I should release some statistics later.

Please do, it'd be interesting.
>> No. 8007
Is it possible to remove a story from your downloader?
>> No. 8008
>>7999

Seconding this.

>>8007

Why would you want to do this?
>> No. 8009
>>8008
Because:
-story is shitty,
-42,
-Obi Wan Kenobi,
-a grue,
-crow slut.
>> No. 8011
>>8008

If requested by the story author, perhaps.

>>8007
Maybe if I have time, but more votes will help make that time...
>> No. 8056
I've been looking for some tool like this for years, and now I found it by accident.

Thank so much.
>> No. 8057
>>8056
Welcome, but "by accident"?
The link to it is right on the front page.
>> No. 8069
Local database of story files, names and authors has been updated to reflect the new story list as of Oct 12.
>> No. 8119
Fagwork's story: "Brazen Accounts of Lovely Ladies' Sensuality" is kind of bugged. I tried to download only the Letty part, and I end up downloading only the Shanghai&Hourai part.
>> No. 8155
>>8119
The reason for this bug is that patchwork switches tripcodes in the middle of the thread, so I would recommend selecting all posters to grab the rest since the downloader does not download stories specified by a post number.
>> No. 8321
Say Cthulhu, can you erase "Mai Arresting and Knocking Armless Idiots" from your story list? Thanks.
>> No. 8322
>>8321
Don't you need ddyk's permission to do that?
>> No. 8324
>>8322
I am ddyk.
>> No. 8325
>>8324
No, anon. You are ddyk.
>> No. 8381
Issue: When downloading large stories, the threads aren't consecutively ordered as they should. I experienced this when downloaded the story " Fragmentation of Memories" as a single .doc file.

Example: Thread 10, T11, T12... T19, T2, T20... T29, T3, T30, T4, T5, T6, T7...

Effect: Fuck.
>> No. 8382
>>8381
May be possible to fix this if threads are renamed T01, T02, and T03 instead of T1, T2, and T3.
>> No. 8416
>>8321
Ask touhou-project.com's admins to delete the story from their database, so that the change is reflected in my database. The reason why I can't do so myself is because I haven't programmed an "exception list" into the downloader. Probably should do that later.

>>8381
Fixed, though lets hope a story doesn't go over a hundred threads so we don't see this error again.

>>8382
Good call.
>> No. 8421
>>8416
There's probably no way they'll do something like that, so I guess I'll just go over it.
>> No. 8717
Are files synchronized with the archives?
In other words, if a story has 3 threads, but if http://www.touhou-project.com/storylist.php links only to two of them, will I get the third thread when using your downloader?
>> No. 8867
>>8717
If the story thread was stored by the downloader, then it will remain in the westisblue database regardless of if the touhou-project story list still has it. It probably won't be visible/downloadable through the app itself however, which means that yes-- the files are "synchronized" with the archives.

Sorry for the late reply. I haven't been around here for the past four months, so here's a simple update.


Delete post []
Password  
Report post
Reason