Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

1

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 09, 2020, 10:31:10 AM »

Oh @Uomoz, what would also be handy is a "debugging" page where a mod author could post a forum link like:
http://fractalsoftworks.com/forum/index.php?topic=11598.0

And the contents of the URL would be scraped & parsed and the relevant info returned to the client so implementers can make sure they have things correct.
Maybe too much work.

It's already like this, it reads, indexes and parses all mod forum pages and sends debug data to an interface I use. So making a debug page is not a problem at all. Right now I can accurately parse fetch and archive 95% of the mods more or less. A few can't be properly downloaded since they don't expose a direct download link. All the others just work

Here's my debug thingie:

2

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 09, 2020, 08:28:35 AM »

By default your mod WON'T be included in the repo. You need to add the repo_crawl = true configuration to allow the repo to interact with your mod.

In this case, I think opt-out is a better choice than opt-in. Inclusion of as many mods as possible is one of the most important aspects of something like this, and I expect many mod authors wouldn't opt in simply because they're unaware or can't be bothered to. On one hand, having to opt out might annoy one or two mod authors, but on the other no one will actually browse for mods on a site that's missing most of them.

That's how I think about it as well. If I had a mod I'd love to have more exposure for it for free (obviously complete credit goes to the modder on the repo; there's absolutely no reference to me as Uomoz or anything anywhere).
But I already received opposed feedback. Should I make a poll? Obviously opt-in is the safer option.

3

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 09, 2020, 07:57:08 AM »

It does fail already, plenty.
Quite a few mods have very poorly formatted jsons, or hrad to download links etc...

But I covered most of the failure cases already.

4

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 09, 2020, 04:41:40 AM »

I really like the idea of a modern tile-based graphical interface for mods, would really enhance the whole scene. That said, like a lot of the other people in the thread I'm concerned about the effort required to maintain multiple forum/nexus/uomozRepo postings (although I don't intend to ever use nexus myself). If your website offers an efficient solution to work with all them I'm all for it 

edit: To add to that, It would be a good idea to have modders contact you (or moderators if that ever happens) to be whitelisted as users that can submit their own packages.

Thanks for the feedback!

The tile graphic was a must have imho

The cool thing of the system is that you basically have just to allow the crawler to go through your mod automatically. I you want you can just skip the whole configuration and just allow the crawler to go through the files. It will automatically detect new versions/changes in the forum post, changes in the images etc. It's a 0 effort thing really. And for those who want they can make so the crawler is smarter about what it finds, but again, is not required!

To submit a package just post a mod in the forum (and add the repo_crawl = true anchor bbcode in the post somewhere) and you are done!

5

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 09, 2020, 03:22:16 AM »

Hey y'all.

I thought about the configs and here's a first draft:

Code

[color=transparent][size=1pt]
[anchor=repo_crawl]true[/anchor] Choose if Starsector Repo should crawl this page or not. [Default]: false. [Optional]: NO
[anchor=repo_category]mod[/anchor] Choose one between totalconversion, mod or utility. [Default]: Mod. [Optional]: YES
[anchor=repo_tags]exampletag1,exampletag2[/anchor] Mod tags separated by a comma. for the full list of possible Tags see http://fractalsoftworks.com/forum/index.php?topic=17654.0. [Default]: N/A. [Optional]: YES
[anchor=repo_title]Title[/anchor] Overrides the title/name found in the mod_json/version files. [Default]: Parsed from version or mod_json file. [Optional]: YES
[anchor=repo_abstract]Lorem ipsum dolor sit.[/anchor] Overrides the description found in the mod_json file. [Default]: Parsed from the mod_json file. [Optional]: YES
[anchor=repo_image]https://www.example.com/image.gif[/anchor] The main mod image (I like a gif here). [Default]: Parses the page for cool images. [Optional]: YES
[anchor=repo_config]data/exampleModConfig.json[/anchor] Specify the config file location of the mod in the main mod folder so it can be edited before downloading it from the repository. [Default]: N/A. [Optional]: YES
[anchor=repo_changelog]Lorem ipsum dolor sit.[/anchor] Specifies the changelog for the latest version. [Default]: N/A. [Optional]: YES
[anchor=repo_link]https://www.example.com/link-dl.zip[/anchor] The current version download link (must be direct). [Default]: Parses the page for probable download links. [Optional]: YES
[/size][/color]

What do you guys think about it? Am I missing something crucial? Consider that once I have a download link I have full access to version/mod_jon file so the information in there is not necessary.

If anyone wants to try configuring this in the mod post so i can test it on a real case, let me know

6

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 06, 2020, 11:13:17 AM »

I found a way to hide stuff in a forum post without it showing mostly and just using this bbc code in the post. In between this line and the code displayed below there's a hidden configuration
https://www.dropbox.com/sh/tcdfwh7smxa7p2k/AABBW-aI6HFXFtGKAhMzl3YKaTotalConversionfalse

Code

[color=transparent][size=1pt][anchor=repo_link]https://www.dropbox.com/sh/tcdfwh7smxa7p2k/AABBW-aI6HFXFtGKAhMzl3YKa[/anchor][anchor=repo_category]TotalConversion[/anchor][anchor=repo_crawl]false[/anchor][/size][/color]

CleverGirl.jpg

I think author defined tags (and tags generally) are the way to go, not personally a big fan of anything crowd sourced.  For tags, you could use a comma separated list like:

Code

[anchor=repo_tags]Faction,Ships[/anchor]

Where the values in tags have to be from an approved set.

When you get a structure/format just PM me and I'll add it to my mods if you want to test!

Thank you so much!

The idea of crow-sourced tags is so even if I can't change the code some time from now the thing keeps on growing and changing by itself. So if a new "type" of tag pops up from a SS patch it's added by the community alone in the system.

That said, I could definitely start from something defined by the author like you mentioned.

I need to think about all the possible parameters I could use to have a faster/easier crawl and post it in the first post.

Thank you so much for the collaboration, if the whole thing crashes and burn and the mod community hates it, it will have been for science and learning

7

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 06, 2020, 07:50:55 AM »

For standard mods, I'd say that major categories are Ship Packs (just adds ships); Factions; Portrait Packs; difficulty changes like Ruthless Sector, Second Wave Options, etc; sector generation changes like Unknown Skies, Adjusted Sector, etc; and then you have a variety of mods that add more things to do like Secrets of the Frontier, terraforming, station building, gate building/reactivation, new industries. The mod index classifies Nexerelin and Varya's Sector as Megamods because they do so many things - I'd like a category name that gives a more specific description than that, but I can't think of one.

Hello! Thanks for the feedback y'all!

Re: Aren't those tags more then categories?

If a mod adds ships and skills for example it could have both those tags.

I'd say a category should be something a mod can only be 1 of without questions. Multiple mods may fill more then one of these categories you describe.

8

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 06, 2020, 06:19:01 AM »

Yeah, my only real concern is having to maintain yet another thing, so I'm also fine with it. Curious to see where it goes and how it develops.

O well with this thing you don't have to do absolutely anything.

Looks amazing!  Any thoughts on categorization / tags?  My instinct is you should request that modders add some JSON file that contains meta-data that you look for.  Even if it's rather spartan now, you'll probably want it eventually.

For tags I was thinking about a system based on users thumbs-upping tags they think are related to the mod in the app itself.
For general categories I need to think about them, suggestions? (I'd say Total Conversions/Mods/Utilities, any others?)

How about JSON embedded in a spoiler tag in the forum post?  That way you don't have to download every zip/7z/rar and pull it apart and guess which one is latest.

As for the JSON content, just off the top of my head:

• Img URL
• Download URL
• compatible version (if it can't be figured out from the post title)
• Mod Version (again if it can't be determined)

I found a way to hide stuff in a forum post without it showing mostly and just using this bbc code in the post. In between this line and the code displayed below there's a hidden configuration

https://www.dropbox.com/sh/tcdfwh7smxa7p2k/AABBW-aI6HFXFtGKAhMzl3YKa
TotalConversion
false

Code

[color=transparent][size=1pt][anchor=repo_link]https://www.dropbox.com/sh/tcdfwh7smxa7p2k/AABBW-aI6HFXFtGKAhMzl3YKa[/anchor][anchor=repo_category]TotalConversion[/anchor][anchor=repo_crawl]false[/anchor][/size][/color]

9

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 05, 2020, 05:53:38 AM »

What about some sort of flag in either the mod_json or the .version file?

There's also another way around this, and it's to hide/display some data in the the forum post so that my cralwler knows some metadata like the official link to the compressed archive, or other flags.

What would you guys prefer, to add some stuff to the mod post or to do edit the mod files (either existing files or a new file)?

For example i'd love a way so the modder can specify the placement of the mod configuration file so a player could configure the mod even before downloading it

10

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: March 04, 2020, 04:10:32 AM »

It's working, mostly.

The thing parses the forum for links from a bunch of different sources (direct link, git, drive etc). Opens the archives, parses mod_json or .version files and try to guess the most accurate data. Automatically update the mod record when the forum post changes.

It's really cool! And it's completely automated. All the information stored in this are publicly available and exactly as found around the forum.

What do you guys think about it?
I'd also love the general opinion about sharing this into the interwebs.. Should every modder be asked for permission for the mirroring of the mod forum post?

11

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: January 14, 2020, 03:47:51 PM »

I'd say that the only true mod repo is Nexus, but I also believe it doesn't really work well with SS, the way is made for multiple games. Feels pretty generic to me.
Anyway, I'll make it so it's so effortless that eventually some modders will try it. The dream is to have modders and players actually use the thing over the other systems (the forum is NOT a good place to store Archives of things IMO, same for Discord and the Nexus can't possibly feature the nuances of the SS community/mod scene) but at the point I'm at already I have already populated the Repo with all the mods in the mod Index. The only thing missing is the DL button linking to the actual DL location, it's a bit of work on procedural url-guessing.
I found ways to guess changelogs pretty decently as well so that's out of the way.

Mass scraping all the posts/discord/possible-locations is out of the way, all mods have multiple files linked everywhere and often are older versions/irrelevant stuff, so it would be REALLY inefficient.

As it is right now it clones the form post in its entirety (with some css magic to make it a bit more pleasing) and it just works.

I was thinking about a "crowd sourced" tagging system where anons could add and remove tags based on general approval and sliders to vote on stuff like "crazy<--->lorefriendly", so it could be easy to find mods similar to your tastes.

Plenty ideas, thanks for all the feedback!

12

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: January 14, 2020, 09:35:31 AM »

That's very useful, to fill info for the mods that are actually present on the Nexus. That said there's no way to programmatically fetch those info without the id, using only the info i can fetch from the mod forum. Mmh.. I could set up a way so the Modder can one-click import all the relevant info in the website from the nexus db just using the api though..

Thanks for the help/feedback

13

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: January 14, 2020, 07:28:26 AM »

Sadly retrieving dl links from the topic seems very hard, simply because there are usually tons of them in the mod first post, and are usually not directly part of a link tag with the Download word in it...



EDIT:

Interestingly enough the whole forum post is easily movable to the Repo as a temporary solution and since it can be crawled on request it will always be updated..

14

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: January 14, 2020, 03:58:35 AM »

That would probably work but would also have a bunch of possible problems related to the automated process (a lot of elements are missing or completely custom depending on the post itself, like changelog or compatibility). Also it would probably look pretty ugly, and would miss a lot of functionalities I'd love to add like tags, batch download, compatibility etc.

That said the idea of scraping the mod forum is pretty interesting as a seeder for the repo.
The Repo could then be automated like that but also accept the modder intervention (after the ownership is verified) so the post it could be properly customized with a premade starting point.

15

General Discussion / Re: Starsector Repo - Starsector Modding Repository

« on: January 13, 2020, 10:02:45 AM »

Thanks for the feedback y'all!

If it isn't a public repository, I can't see it taking off in any shape or form. First because it would be an insane amount of work to maintain up to date (since we now have mod updating every day) and it would create the conundrum of modders wanting to keep control over their releases, yet not willing to spend time updating their mod on yet another platform.

I mean, the base idea has many merits but it would require some more features to be worthwhile I think. Some basic mod management for example (Update all mods, Safe update mods [no save breaking update] ), one click download+unpack maybe? Basically Nexus but without the login part.

It will definitely be public and every mod i put in myself (that will be only at the beginning ideally) will be done after asking the modder permission. I intend to make it the best place to upload the mods, even if it doesn't work in the end. Functions like "batch dl with save compatibility" are definitely planned. I plan to add a lot of functions that we usually do browsing files in remote folders in a single checkout page. For example you could have a package of mods you like the most bundled together with all the configs already set up to your preference and downloadable every time you want. Or having different packages and one could also check for mod compatibility on the go and possibly modders could inform players for mod to mod problems or suggest configurations.