scaleopk.blogg.se

Set to true to disable scraping threads on messages that are in channels or conversations.I recommend you start without running headless mode with one conversation or channel since you can see clearer if the collect scraper is working or not. Helpful for scraping long channel/conversation histories, since the browser runs with a larger vertical viewport so can scrape it larger batches at a time.Set to true to scrape with the browser in headless mode.For a very active channel, it could take 60 seconds to scroll up half a year then ~20min to scrape it. Useful when scraping channels with a long history but don't need to scrape it all. A timeout in seconds for when to stop scrolling up the channel history and start scraping posts.The channel or the conversation must be in the list of channels or DMs in the left sidebar before running the collect script.For example, if a name tag includes an emoji, you can only write the part of the name tag without it and it should work. The name doesn't need to be an exact match, it must only match part of the name.It's the name you see under "channels" side tab in Slack. Set CHANNEL_NAMES to scrape a public or private channels.The value is the name tag of the person or group chat name as is written under "Direct Messages" in Slack. Set CONVERSATION_NAMES to scrape a DM or group chat.You can escape a double quote in a string in JSON like this: The array elements are double quoted and the last element doesn't have a trailing comma. The collect script will scrape the list of conversations first, then the list of channels.You must set one of CONVERSATION_NAMES or CHANNEL_NAMES or both. SLACK_EMAIL and SLACK_PASSWORD are credentials used to login into the workspace.Note environment variables are set without quotes. SLACK_WORKSPACE_URL must be the URL you login to the workspace not.SLACK_WORKSPACE_URL, SLACK_EMAIL and SLACK_PASSWORD are required. Then modify following environment variables in. example.env file in the project root folder and rename it to. Run npm install to install the dependencies.It's helpful for saving information from a channel or DM without needing to ask a workspace administrator to export the data.įor example, if you're in the process of leaving your current company to join another, this tool is a great way to archive everything you've said and done on Slack. Instead, it logins to your Slack account and uses that to access the channel or DM. It doesn't depend on installing an app in the Slack workspace or aquiring an API key.

It uses Puppeteer headless browser for loading and interacting with Slack. A web scraper that navigates to a Slack workspace and saves the posts and threads of a given channel or DM.